











Measuring Public Attitudes 





Issue Editor: ANGUS CAMPBELL 








CONTENTS OF ISSUE 


Attitude Research and Government 
WALDEMAR A. NIELSEN 


The Uses of Interview Surveys in Federal Administration 
ANGUS CAMPBELL 


Public Opinion Polls and Democratic Leadership 
DORWIN CARTWRIGHT 


The Interpretation of Survey Findings 
DANIEL KATZ 


How Surveys are Made 
ELEANOR E. MACCOBY and ROBERT R. HOLT 


A Summing-up 
ANGUS CAMPBELL 


} 











May, 1946 





THE JOURNAL OF SOCIAL ISSUES 
published quarterly by 
THE SOCIETY FOR THE PSYCHOLOGICAL STUDY 
OF SOCIAL ISSUES 
A Division of 
THE AMERICAN PSYCHOLOGICAL ASSOCIATION 
(For subscription rates and price of single copies see last page) 


STATEMENT OF PURPOSE 


The SPSSI is a group of several hundred social psychologists and allied social scien- 
tists with a particular interest in research on the psychological aspects of key social issues 
that arise in the: course of man’s living with man. This publication is tars a strictly 
technical research bulletin as the main organ of the Society because of a belief that there 
set need for the social scientist to communicate his most applicable research findings 

his most mature professional interpretations to the applied social scientists in govern- 
ment, industry, education, social work, group work, religious education, etc. 

In short the Society believes it has an obligation to help get scientific knowledge into 
the bloodstream of applied social action by building as many bri as possible between 
theory and practice in group, community, national, and international living. This journal is 
one experiment in this direction. As we see it, the members of the Society must make every 
effort to communicate in a ey Pra professional manner their findings and interpre- 
tations without setting up. too di t barriers of technical language and statistics on the 
one but ce Serums ing from undisciplined thinking and patronizing populariza- 
tion on the other 

‘We feel that applied scientists in return have the aye to make us more and more 
aware of their key problems on which fact-finding is led, and to criticize the style and 
contents of the in doing the job it has set out to do. At every step such an enter- 
prise as this will have to be a cooperative one between the scientist and the actionist. We 
invite you to join in these steps. 


Business address: een. orders, and business communications should be sent to 


ASSOCIATION 347 Madison Avenue, New York 17, New York. 
Editorial address: ions and inquiries concerning editorial contents should be sent to: 
Ronald Lippitt, Center for Group Dynamics, Massachusetts Institute of 
, Cambridge, Mass. 


Society addvess: Request for information about the Society and applications for membership 
be sent to Dr. Daniel Katz, Secretary-Treasurer, Society. for the Psychological 
Study:of Social Issues, 36 Berkshire Road, Rockviile Centre, New York. 


Executive Editorial Committee: Ronald Lippitt, Massachusetts Institute of Technol 


Clark University; Ruth B 
Bonney, North Texas State T: 














Preface 


This issue takes the reader to the growing edge of one of the most exciting 
frontiers of applied social science. All of us have recognized the danger to 
democracy that lies in the widening gap of non-communication between the in- 
creasing hierarchy of leaders and experts on the one hand and the body of citizens 
of organizational members on the other. More and more the citizen has felt that 
government is ‘‘off there”, a thing apart from him, with no interest in his welfare, 
and beyond his power to influence. A sizeable proportion of elected and ap- 


pointed leaders have also felt keenly the disadvantages of this loss of communica- 
tion. 


With mixed feelings of welcome and cautious distrust both victims of this 
dilemma, the leaders and the led, have watched the development by social scien- 
tists of new methods of public attitude and opinion measurement. Just how valid 
are the results from these small samples of people that are talked to? Whose 
interests are being and will be served by the growing volume of polls? 


Although a large number of specialized research organizations now follow 
the trail blazed by Dr. George Gallup and offer research services in opinion poll- 
ing and attitude sampling, no other has had so much experience serving as the 
communication middleman between the governed and the governing as the Divis- 
ion of Program Surveys of the U.S. Department of Agriculture. Under the 
pioneering leadership of Dr. Rensis Likert, president-elect of the Society for the 
Psychological Study of Social Issues, the Division’s services have been used widely 
in governmental operations. For this reason the Editor turned to Dr. Campbell 
and his colleagues who have been associated with the development of the research 
program of the Division of Program Surveys with a request to set down in the 
short space of one issue as integrated a story as possible of their experiences as 
scientific middlemen. 


The Editor is well aware that this issue is not an exhaustive or representative 
treatment of the various technical points of view about attitude measurement 
procedure. That is not the purpose of the issue. A selected bibliography is ap- 
pended for those who wish to read further in this direction. 


We believe the reader will find exciting vistas into the democracy of the 
future opened up to him by these informative descriptions of the practical possi- 
bilities of attitude measurement and by the thoughtful interpretations of the 
proper place of this part of social science in a democratic society. 


RONALD Lippitt, 
General Editor 











Attitude Research and Government 


WALDEMAR A. NIELSEN 


The application of interview surveys! as a tool of public administration is 
a relatively recent development in the United States. Although the technique 
has been accepted for many years as a useful device for large-scale management 
in the field of private industry, question as to the propriety of its sponsorship by 
government is still sometimes raised. There are those who praise it as a basic 
line of attack on the great problem of “democratizing” administrative policy- 
making. And there are those who condemn it as an instrument for destroying 
the American two-party system. 


During the war the study of public attitudes and opinions was carried on 
both in the United States and in the totalitarian states of Germany and Japan. 
Comparisons are now possible between the methods used, the purposes served, 
and the results achieved in the three countries which demonstrate the funda- 
mentally democratic character of the technique and emphasize certain traits of 
totalitarian governments which are commonly suspected but seldom specifically 
documented. 


Since the turn of the century, and especially in the last fifteen years, public 
administration in the United States has been characterized by the following 
trends in its scope and function: increased reliance on administrative planning 
within the boundaries of broad legislative directives; a shift of administrative 
responsibility upward from small local units to larger regional and national ones ; 
a progressive transfer of authority over economic affairs from private to public 
hands; and a grand expansion in the scale of operations. 


These developments have affected ordinary citizens, as well as government 
officials, very directly. The citizen has been more and more in a position of 
having his affairs subject to the acts and decisions of gigantic organizations with 
which he had no adequate contact and over which he seemed to have progressively 
less control. The official, charged with responsibility for a program which de- 
pended ultimately for its success upon the understanding and cooperation of the 





1The term “interview survey” is used here and elsewhere in this issue to refer to a 
more inclusive type of study than is suggested by the term “attitude survey”. Sample sur- 
veys based on personal interviews are commonly designed to secure a wide range of data 
bearing on public attitudes, expectations, information, and activities, as well as many items 
of a socio-economic character regarding each respondent. Unfortunately, there is no single 
term which adequately defines this cluster of interrelated facts. 
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general public, has felt a growing sense of remoteness from the individual citizens 
whose welfare he was supposed to serve. Thus, there has been need, felt both 
by officials and the public, for a means of bridging the gap between them, for a 
means of acquainting administrators with the preferences, requirements, hopes, 
fears, and complaints of the people whose lives their programs affected. 


To bridge this gap, government officials in the executive branch prior to 
1936 relied exclusively on inexact, haphazard methods of determining public 
reactions. They collected the reports of field agents of their own organizations 
who were in direct contact with the public, analyzed letters of complaint and 
praise which reached their offices, studied comments in the press and over the 
radio, observed developments in Congress, and were subjected to the appraisals 
of the representatives of interest groups who visited them as special pleaders. 
As individuals, they included their impressions gathered from overheard conver- 
sations in street cars, dinner-table and golf course gossip, and the statements of 
trusted friends, in making their judgments about the state of public opinion. 


The use of scientific methods of attitude research in government began in 
1936, when Henry A. Wallace, then Secretary of Agriculture, at the suggestion 
of Mr. M. L. Wilson, authorized the systematic study of farmers’ experience with 
the agricultural programs. In 1939 the research unit was expanded somewhat 
and transferred to the central planning agency of the department, the Bureau of 
Agricultural Economics, which stood in a staff relationship to the Secretary. 
From that time until general preparations for war were being made in the latter 
part of 1941, the Division of Program Surveys, as the unit was named, clarified 
its functions, developed its methods, and gradually increased in size. 


Although the division was the only agency in government engaged in atti- 
tude research, it never approached either in number of employees nor magnitude 
of expenditures the size of any of the major fact-finding bureaus. At no time prior 
to the hectic “defense” period did federal expenditures for attitude research 
exceed one hundred thousand dollars per year. During these same years private 
businesses were spending millions of dollars annually to find out consumer prefer- 
ences about their products and consumer attitudes toward their services. Con- 
currently, the techniques of straw-polling and election prediction were being 
applied to a wider variety of “‘headline issues” which developed between elec- 
tions, and social scientists in all fields were conducting basic research on economic, 
social, and political attitudes. 


During the period of the recent war, the tendencies of government to grow 
in size and proliferate in function were, of course, sharply accentuated. To a 
greater extent than ever before the physical and human resources of the nation 
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were mobilized for a single effort, and it became more important and more diffi- 
cult than ever to maintain the link between administrators and citizens. The 
OPA touched directly the intimate details of the citizen’s livelihood. The War 
Bond program clamoured for his pay-check. The War Food program was a factor 
in his farming. The WPB was a partner in his business. Without his coopera- 
tion, his acceptance and understanding of the purposes of the programs, over- 
whelming black market activity, violations, and finally political opposition would 
have destroyed them. 


In every phase, the war directors had to rely on the support of the people 
either on a completely voluntary basis, or on the basis of arrangements com- 
pounded of economic punishment and reward, but depending upon a considerable 
degree of participation by the public. The sweeping nature of economic and 
social programs makes it futile to denounce non-cooperators as ‘‘law-breakers,” 
“offenders,” and “violators.” In a democratic system, compulsion has a very 
limited use; programs and administrators have to stand for re-election, in a sense, 
just as members of the legislature. 


That the scientific sampling of public attitudes should have been accepted 
during this period as a proper staff function of the administrative branch of 
government, and that it should have enjoyed a thriving growth, was to be ex- 
pected. About the time of Pearl Harbor, attitude and opinion researchers from 
all fields were drawn into government service. Nearly every major war agency at 
some time during the war was served by the attitude research organizations which 
developed, and most of the governmental agencies collecting statistical data in 
the field at one time or another ventured into the area of attitude research. Two 
agencies, the Division of Program Surveys and the Surveys Division in the Office 
of War Information, became the principal service units to the civilian war 
agencies. The Research Branch, Information and Education Services, Army 
Service Forces, served the Army. The Office of Civilian Requirements conducted 
studies for the War Production Board. 


Attitude research during these years marked off and established an area of 
accepted function for itself in government.! Not so much by design as by adapta- 
tion to administrative needs as they became apparent, the list of services per- 
formed grew until it covered most of the major problems of large-scale manage- 
ment. Despite its versatility, however, attitude research was used principally on 
problems connected with the relations of government organizations and their 
external publics—the segments of population affected by their programs. Within 





1For a discussion of the range of administrative problems which might be subject to 
attitude research, see David B. Truman’s “Public Opinion Research as a Tool of Public 
Administration,” Pub. Admin. Review, Vol. 5, No. 1, 1945, pp. 62-72. 
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this area the greatest application was to informational problems and problems 
associated with planning and appraisal of programs. 


As in the case of private business, a number of uses were found for it in 
relation to internal problems also, particularly personnel. The War Depart- 
ment periodically surveyed the members of the Army for their opinions about 
food, clothing, entertainment, discharge plans, promotion practices, the deficien- 
cies of leadership, and the administration of discipline. The Navy Department, 
though more reluctant to endorse the work, carried on certain attitude surveys 
among its thousands of civilian workers in Navy yards and docks, and in despera- 
tion, after the end of the war, resorted to surveys to provide guidance to manage- 
ment in developing a voluntary recruiting program, particularly for officers. 


Whether evaluated in dollars-and-cents terms of efficient, useful public 
administration, or in terms of its larger relationship to the problems of demo- 
cratizing modern bureaucratic government, the research method proved its worth 
during the strenuous years of the war. 


The exigencies of war fostered the development of attitude research in the 
United States as a tool which administrators could use in adapting their policies 
to public wishes. The form which the development took reflected the basic 
assumption that government policy must be in accord with the views of the public. 
In Germany, and Japan,? although the governments were faced with problems of 
the same nature and of even greater urgency as the war progressed, public opinion 
was regarded from a very different standpoint. Government policy was made at 
the top, and it was the task of leaders either to inspire or to compel the popula- 
tion to follow. The study of opinion, therefore, was carried on to determine the 
extent to which the population was filling its obligation of obedience, and the 
extent to which officials had been successful in “leading” the people. 


Essentially, when public opinion was consulted, the people were on trial, 
not the programs. The methods used were those of investigation, not research, 
and the police and the propaganda ministries did the work. “Subversive thoughts 
and actions” were extensively reported, usually under the general heading of 
“morale.” In Japan, for example, whatever information the officials obtained 
regarding public attitudes toward their programs they were compelled to gather 





1These are described in detail in the succeeding article in this issue. 


2In this section primary reference will be made to Japanese experience, inasmuch as 
very little has been published on the characteristics of its dictatorship. Reference will 
frequently be made, however, to the German situation. 
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from their own personal sources, or to receive incidentally from police reports 
conducted primarily for police purposes. 


An official in the Japanese Home Ministry: 


“I got reports on incidents from all over the country through the Home Minis- 
7 They were only fragmentary incidents, but they gave me an idea of the trend 
of what was happening. . . . I had to estimate the thought or reason behind the 
incidents. I read the Tokyo newspapers and compared the facts with reports of 
actual incidents that came into the Home Ministry, and by that I came to conclusions 
about the trends. From the papers I read letters to the editor, editorials, and small 
incidents which were reported, and these were vague markers of public opinion, 
but all I had to use as a gauge.” 


The principal police reporting systems in Japan were the Kempei Tai, or 
military police, and the Special Higher Police of the Home Ministry, commonly 
called the “thought police.” Both of these organizations reported continuously 
on developments in the country, and exercised a very intimate control over what 
was happening. However adequate their domestic intelligence reports may have 

‘been from a police viewpoint, they were shot through with shortcomings as re- 
search. The police investigators relied entirely on the devices of espionage, 
failed entirely to sample except on the most crude geographical basis, and did 
not attempt to treat opinion data quantitatively. Neither were the problems of 
warping and bias given any systematic consideration.? 


In the latter months of the war in Japan, when national solidarity behind 
the war effort was rapidly breaking up and doubts and uncertainty were running 
rife, some half-hearted attempts were made to investigate what people thought 
of various gove:nment programs through more reliable sources than police agents. 
One was done by an ad hoc committee, set up within the government and report- 
ing directly to the Cabinet. Two others were done on government request by the 
local representatives of the government-controlled news association, Domei. At 
best, however, the information which policy-makers had available for evaluating 
their programs in terms of public acceptance was extremely weak and subject to 
error. 


The German case was quite similar. In a report of the U. S. Strategic Bomb- 
ing Survey, the following description is given:* 
“German authorities, concerned with the state of qed morale, maintained 


an extensive intelligence service on the home front, both for their general informa- 
tion and as a specific guide to the propaganda efforts. Many officials within the 


1To what extent these weaknesses were due to the fact that a police organization carried 
on the work is difficult to judge. There probably was no research organization in the 
country capable of doing an adequate job. The general absence of statistical sophistication 
is suggested by the fact that in Tokyo Imperial University, the nation’s leading institution 
of higher education, not one course in statistics was taught. 


2"The Effects of Bombing on German Morale,” U.S. Strategic Bombing Survey, Vol- 
ume I, Part II, unpublished, pp. i-iii. 
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hierarchy of the civil and Party administration were regularly engaged in writing 
morale reports. The Security Service, the Propaganda Ministry, and other govern- 
mental departments all had their agents throughout the population furnishing mate- 
rial for these intelligence reports. 


“The official German Morale Reports give a picture of German morale as seen 
by officials during the war. They are of great value as a secondary source of in- 
formation, both for their account of morale and for the insight they furnish into 
German official thinking. The reports are limited, however, in that the Germans 
did not avail themselves of modern scientific techniques for the study of popular 
thought and feeling. Quantitative controls, sampling methods, and research design 
were completely lacking in the collection and interpretation of material for those 
reports. Hence the German intelligence reports cannot be used to obtain estimates 
of the degree and extent of weak or strong points in morale with any precision. They 
show the problems of maintaining war morale in a totalitarian state and they give 
illustrations of the qualitative aspects of these problems, but they do not indicate 
in any quantitative manner the relative extent or depth of the various elements of 
morale. In addition to the defects in the collection of data, the Morale Reports suffer 
from two possible sources of subjective bias: 


1. The tendency for the reporting official to give his superiors the type of 
material they wanted; and 


2. The tendency for the reporter to exaggerate the importance of the attitudes 
and beliefs of his own social group.” 

In Japan it was in the interpretation and use of information about public 
attitudes and morale that the precepts of science were most outraged. In a totali- 
tarian system it could not be expected, of course, that manifest needs and wishes 
of the population would be regarded by the war managers as admonitory or even 
advisory. In Japan, as in Germany, the desires and feelings of the population 
were ignored, exploited, manipulated, punished, or praised—depending on their 
accidental coincidence or conflict with the ‘‘party line” of the governing elite of 
the moment. In the face of public discontent, for example, the first impulse of 
Japanese officials was to suppress it or change it rather than be guided by it. 


An official of the Home Ministry: 


“.. . When farmers began keeping rice from the government warehouses, the 
government increased its supervision of this. When absenteeism and work slow- 
downs increased, the government made absenteeism without good excuse a punish- 
able offense and began to hold managers responsible for actions of their employees. 
This was in June, 1945. There was no method of alleviating the people’s discon- 
tent. The government merely added more pressure, held more lectures, distributed 
more propaganda.” 


Throughout, public opinion was rather judged than listened to. Police off- 
cials applied a standard of morality to it, and classified it as “‘good” or “bad,” 
and if the latter, their interest was to punish the individuals concerned, or sup- 
press the information. Military men, and especially Premier Tojo, simply dis- 
missed such information as unworthy of consideration. Propagandists followed 
the results in order to guide their efforts in “correcting” the public’s views. Some 
officials watched what appeared to be the shifts in public sentiment in order to 
estimate the probability of disruptive uprisings. Others took any information 
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that came to hand and used it to sell their own views if it were appropriate, or 
denounced it if it were not. Only in rare instances was public opinion taken as a 
guide to action directly, and as a valid indicator of public needs. Even the de- 
cision to surrender was made with only secondary reference to the state of public 
morale; in the opinion of the highest officials, the country was not yet on the 
verge of revolt, and therefore public opinion had not reached the point where it 
deserved respect. 


Moreover, until very late in the war, what little was known about public 
opinion was not circulated among officials in the government. The disease of 
dictatorship, the suppression by force of criticism and suggestion from below, 
infected not only relations between the government and the people in Japan and 
Germany, but also relations between persons in the various divisions of govern- 
ment itself. Feelings of distrust, insecurity, suspicion, and self-defensiveness 
pervaded the administrative structures, and resulted in the common concealment 
of reports, of personal opinions, and intentions. Mr. James Cavin of the United 
States Department of Agriculture, in describing his experience in Germany in 
the first months of the Occupation, has stated that the examination of a large 
body of administrative memoranda, reports, and inter-office communications did 
not yield any instance in which a lower official openly criticized or took exception 
to the work or opinions of someone higher. The situation in Japan seems to have 
been quite similar. 


An official of the Police Affairs Bureau, Home Ministry: 


“I do not think these reports (on the state of morale) were sent anywhere else 
until toward the end of the war, when they were given to various ministries and 
government agencies. I think that the reason the reports were not given to the Army 
and Navy and other agencies of the government in the earlier days of the war was 
that they treated mostly with what the people thought about the war. The people 
did not seem to have much confidence in victory, and if such reports were sent from 
my office to the armed forces and the government, the people who read them would 
only become angered by them.” 

In Japan, all officers of the government, from the lowest to the highest, were 
compelled to engage in the practice of pretense. Despite the shifts in cabinet 
which occurred, the weakening in the control of the militarists over government 
policy, and the desperate plight of Japanese forces, the mask of appearances which 
the public saw changed very little until the end. Lower officials continued to go 
through the motions of their work. The Diet regularly expressed its support for 
the struggle. Even Baron Suzuki, who was known within the highest circles to 
be a leader of the anti-war group, felt obliged to exhort the populace to make 
every sacrifice for the war as soon as he was appointed to the Premiership. Imme- 
diately afterwards, however, he went to work on the svb rosa negotiations for a 


surrender. The Emperor himself spoke to the Diet about the importance of 
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winning the war at a time when he was conferring with his advisers on the ques- 
tion of how Japan could be “withdrawn” from the war. 


Baron Suzuki, former Premier: 


“I was naturally in a very difficult position because on the one hand I had to 
carry out, to the best of my ability, the instructions more or less given me by the 
Emperor to arrange for a conclusion of the war, whereas, if anyone heard of this I 
would naturally have been attacked and probably killed by people opposed to such 
a policy, so that on the one hand, as far as Imperial General Headquarters was con- 
cerned, the government in general, and the people, I had to advocate an increase in 
the war effort and determination to fight on, whereas through diplomatic channels 


and any means available, I had to try to negotiate with other countries to stop the 
war.” 


An official of the School Bureau of the Ministry of Education: 


“Suzuki concealed his intentions, and among intellectuals it was great sport to 
discuss what his real intentions were—what he actually meant by the statements 
which he made.” 


Conceivably, of course, it is not impossible to maintain mechanisms of inter- 
nal self-criticism in a totalitarian situation. However, in Japan that did not 
happen, and in Germany, despite a most careful attempt to develop an agency 
within the official heirarchy with freedom to criticise, only very partial success was 
achieved. Under Heinrich Himmler, as part of the Elite Guard organization, a 
special Security Service existed. Responsible Security Service personnel were re- 
cruited from the most zealous and reliable members of the Party, and their reports 
could afford to be quite frank. Nevertheless, the function served remained essen- 
tially a police one and the emphasis was upon control of the population and 
control of the bureaucracy. Suggestions for modifying onerous policies to coin- 
cide with popular desires were not made, and reports which were prepared were 
not distributed among the operating officials of government for their information 
but rather were closely guarded within the police system itself. 


The first, and greatest, loss as a result of this missing link between the gov- 
ernment and the people in both countries was the progressive failure of adminis- 
trative programs. Initial mistakes were persisted in; inequities were either inten- 
tional or never discovered, but they worked their effects in either case. Energies 
that might have been devoted to winning the war were consumed in unnecessary 
frustration or spent in spreading dissatisfaction. Japanese farmers, for instance, 
objected to the terms under which they were compelled to produce and sell their 
crops, and they sabotaged the food program extensively. Manpower mobilization 
decrees were flaunted and undercut. At almost every point where the success of 
programs depended on the cooperation of the mass of the population, they were 
hindered by apathy, disillusionment, or resentment. Instead of unifying the 
nation, the war effort, partly because of the manner of its direction, tended to 
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fractionate it and separate the government from the people. As one Japanese 
observer stated: 
“The only thing that unity was developed in was in abandon—best expressed 
by the Japanese phrase, ‘Shigata ga nai,’ ‘It can’t be helped’.” 

In the absence of information about what was happening, administrators 
were at the mercy of their wishes, hopes, and fears. Information officials con- 
tinued to devise little schemes intended to delude the public about the effects of 
air raids, for example, at a time when the public had long since lost its confidence 
in leaders, the government, and all official news announcements. And, as com- 
monly happens in the absence of adequate information about public opinion, 
rumors tended to circulate among official circles, and reactions in some instances 
were extreme. 


In February, of 1945, Prince Konoye presented to the Emperor a memo- 
randum of his views which might serve as an epigraph of the end results of 
dictatorship—panic and trepidation among the rulers and profound unrest among 
the people: 


“What we have to fear, therefore, is not so much a defeat as a Communist 
revolution which might take place in the event of defeat .. . 


“Conditions, internal as well as external, point to the danger of such revolu- 
tion . 

“With regard to internal affairs, potentially dangerous factors include the rapid 
deterioration of the people’s living conditions and the increase in the voice of the 
labor classes. . 


Whatever may /~ the virtues of “efficiency” which have been claimed for 
totalitarian methods, these “inefficiencies” seem to be characteristic: whenever a 
divergence develops between official policy and popular desires the population 
becomes dangerous in the eyes of the dictators; inasmuch as the regular adjust- 
ment of governmental policy to shifts in public sentiment is denied, criticism 
from lower levels of society upward, and from lower levels of officialdom upward, 
becomes impossible, and conciliatory means for re-directing policy are exchanged 
for methods of control of the population by force; in this situation fear and 
distrust spread, disaffection develops between the government and the popula- 
tion, and the administrative effectiveness of programs breaks down in the face of 
widespread apathy, indifference, or resistance. It does not appear, however, that 
the results of totalitarianism go beyond this. Until the end police controls over 
the populations remained in effect in both Germany and Japan. 


This sequence is perhaps normal and even predictable. The particular point 
of recapitulating it for purposes of this paper is that in such a situation, despite 
the utmost need for public attitude research as it has been carried on in the 
United States, it did not develop. Apparently, by its nature, and by the nature 
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of the totalitarian state, such compromises as would have been necessary to fit the 
one to the other were impossible. 


The evidence which is available suggests that for very basic reasons totali- 
tarian governments cannot use attitude research for “good” purposes, i.e. adjust- 
ing policy to the nation’s desires. Moreover, they have the greatest difficulty in 
using the technique even for purely expedient purposes, i.e. achieving some ob- 
jective appraisal of the effectiveness and adequacy of their own programs. It 
appears that the useful application of attitude research by government requires 
considerable tolerance for criticism within the bureaucratic structure, and mutual 
confidence between the governors of the state and the people. These premises 
did not obtain in Germany and Japan, and opinion research took the degenerate 
form of police spying. In the United States, however, attitude research flourished 
during the war and proved its usefulness. 


Now that the war is over, the question arises, “Where do we go from 
here?” To assume that its war record will guarantee an expanding future for 
attitude research in American government is to assume a simple relationship be- 
tween virtue and reward which does not in fact exist. It may merely be given its 
honorable discharge. 


Since the surrender of Japan, the OWI Surveys Division and the Office of 
Civilian Requirements unit in WPB have been disbanded, the Army organization 
has been greatly reduced in size, and the appropriation of the Division of Pro- 
gram Surveys in the Department of Agriculture has been cut to its pre-war level. 
Two extensive but temporary survey organizations have been created to study the 
effects upon civilian morale of the bombing of Germany and Japan, but in Ger- 
many, where arrangements were made to continue the survey group as an admin- 
istrative service to the occupation commander, the proposal was rejected.! For 
the present, at least, the work in government will be restricted. 


A small study conducted recently? among legislators and adininistrators in 
the federal Government suggests some of the reasons why, once the requirements 
of national defense have passed, attitude research has been caused to wither 
unduly. Mr. Kriesberg collected the opinions of a number of federal officials 





1The Information Control Division of American Military Government has recently 
established a limited attitude research organization. 


2Kriesberg, M., “What Congressmen and Administrators Think of the Polls,” Pub. 
Opin. Quarterly, Fall, 1945, pp. 333-337. 
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and Congressmen about the propriety and desirability of government interest in 
polls, with these principal results: 


a. Less than a majority of either group favors official government interest 
in public opinion surveys, regulatory or otherwise; 

b. Administrators at the policy level are more favorable toward polls than 
are Congressmen; 

c. The principal criticism of polls offered is that they are “inaccurate.” 


In discussing the probable ‘““Why’’ of his findings, Mr. Kriesberg says: 


“The relative indifference or hostility of Congressmen to polling seems to 
spring largely from three factors: 1) They feel that polls are challenging their 
prerogatives of interpreting the public will; 2) They are tied by the elective process 
to the more vocal and influential constituents in their state or district (whence 
perhaps arises their feeling that representative polls are “‘inaccurate’”); and 3) They 
are relatively unfamiliar with polling methods and therefore do not have confidence 
in polling results. . . . Administrators, on the other hand, see in the polls one of the 
few possible ways to get public participation in the administrative process and to 
check the power of pressure groups which are so often the bane of a bureau. Admin- 
istrators are probably the more inclined to accept the polls because they have seen 
polling operations at close hand. They frequently think of polls in terms of the 
ones which their own Department has used, and thus as something which they con- 
sider both useful and reasonably accurate.” 


A partial failure of attitude researchers themselves to ‘“‘build understanding 
and acceptance among the people affected by their program’’—namely, the Con- 
gress and top administrators—is evident. Nevertheless, their task of education 
and salesmanship has not been easy. The polling of public opinion is likely to be 
viewed negatively rather than positively by any persons who possibly feel them- 
selves to be on the defensive. It can be considered as a method of perpetuating 
a party in office by means of improving its administration of programs; as an aid 
to the propagandizing of the public through its usefulness in the direction of 
informational programs; as a threat to the prerogatives of Congress as the reflec- 
tor of public opinion; as an offense to those lobbyists and public relations con- 
sultants whose profession it is to interpret and define public opinion. It can be 
called a system of “thought police” and political organization. It can be said, 
lastly, to encroach upon the provinces of administrative and political leadership. 


Each of these derogatory contentions can be countered. That a party in office 
should take advantage of every means to improve the efficiency and democratic 
character of its administration is hardly a development to be “viewed with alarm.” 
Whether research is used to facilitate a “propaganda” program or an “‘educa- 
tional” one depends upon the intentions and the integrity of policy-makers; that 
the technique can be used for wrongful purposes does not alter the fact that it 
has been used for purposes of better government with great success. The research 
technique constitutes an attempt to withdraw from the jurisdiction of intuition 
and guess-work certain questions which relate to administrative decisions and 
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which are subject to study objectively; only if one maintains that official leader- 
ship should be exercised without benefit of reliable factual information can atti- 
tude research be criticised as a threat to policy-makers’ prerogatives. 


The strong case which can be made for attitude research in government has 
not always cogently and courageously been made. Nevertheless, there are good 
reasons for expecting that the current decline in the amount of work being done 
is only temporary. The growth in size and function of government provides an 
expanding potential market for such research. The studies which have been done 
have been useful, and in the course of them administrators have been educated 
to the possibilities of the technique. They therefore can be expected increasingly 
to call for further studies whenever their programs are in serious need of ap- 
praisal in terms of public response. Finally, the curtailment of attitude research 
has largely accompanied the curtailment of all functions of the war agencies. It 
is likely that the new agencies of peace and reconversion, such as the Veterans 
Administration and the National Housing Agency, will take up the work in the 
course of time. 


At any given moment, however, the extent to which it will be supported by 
the government will depend to a considerable degree upon general political 
circumstances—upon the degree to which urgent national problems require 
administrators to abandon unprecise, half-way methods of determining public 
reactions; upon the intellectual climate in high officialdom and the extent to 
which administrators are brave enough to have data collected about their mistakes ; 
and upon the extent to which economy-minded Congressmen recognize that im- 
proving the efficiency of government through the introduction of tested research 
methods may be a surer means of saving money than indiscriminate budget- 
hacking. In one sense, the contest of attitude research for recognition and accep- 
tance is only one phase of the contest of social science in general; atomic scien- 
tists have been hobbled with security regulations, but social scientists have long 
had to struggle against prejudices, misunderstanding, and vested political inter- 
ests. 


The fact, however, that opposition to attitude research has not been sufficient 
to block its acceptance in time of crisis in itself reveals a fundamentally important 
characteristic of American government. When it is. recalled that attitude research 
was actually used on a wide scale, and that the results of the work did influence 
major policy decisions, thus serving to “democratize” administrative policy 
making, then a happy contrast becomes clear between American war experience 
and that of totalitarian government in Japan and Germany. The American oppor- 
tunity to make further use of survey techniques in the difficult problems of peace 
should not be lost. 
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The Uses of Interview Surveys in Federal Administration 


ANGuS CAMPBELL 


Government administrators have only recently begun to exploit the poten- 
tialities of research based on interview surveys. Many federal officials have not 
felt the need of surveys of public thinking and action, believing either that a 
knowledge of public response is not necessary to the success of their programs or 
that they receive adequate information through other channels. Some have found 
their sources of information inadequate but have not recognized the ways in which 
survey research could be useful to them. 


As the previous article points out, the war years brought about a great in- 
crease in interest among government officials in the applications of survey tech- 
niques to problems of federal administration. The urgent necessity of assessing 
changing public attitudes and activities during the war led to a considerable 
extension of fact-finding services. Government surveyers were called upon to 
study a great variety of wartime problems and they furnished many valuable data 
which could not have been otherwise obtained. 


The uses government administrators have made of interview surveys during 
the last five years have varied according to their specific needs for information. 
The subject matter covered has differed from agency to agency and from time to 
time as federal programs have changed. However, the major objectives which 
these studies have been intended to answer may be summarized under six general 
questions. 


How Well is the Public Informed? 


Surveys of public information vary from the merely interesting to the speci- 
fically practical. The former type of survey is exemplified by the reports issued 
occasionally by the commercial polling agencies, demonstrating the extent of 
public ignorance of presumably well-known facts. It has been shown recently, 
for example, that only a small percentage of the population have any clear idea 
of what the Bill of Rights is. Similar surveys have shown widespread inability to 
identify public figures whose names are before the reading public constantly. 
Surveys of this character are revealing in regard to the general state of public 
enlightenment, but they are not ordinarily directed toward the solution of specific 
administrative problems. They have a certain interest to the followers of news- 
paper poll reports, but they have no bearing on actual problems of policy. Such 
studies are never done by federal agencies. 
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However, there are two types of survey done by government agencies, 
intended to find out how well people are informed, which have definite adminis- 
trative usefulness. The first of these are surveys designed to evaluate information 
campaigns. 


Assessing Information Campaigns 


During the last few years the federal government has been the largest single 
purveyor of information in the country. Its promotional campaigns have been 
conducted on an unprecedented scale and have involved a very sizable investment 
of money and manpower. It has been essential that some method of assessing 
the success of these programs be available. 


The complexity and breadth of the typical federal information campaign 
make the evaluation of the procedures used very difficult. It cannot be done by 
counting the number of posters, news releases, and radio performances that have 
been put before the public and estimating from the total the proportion of the 
population that has been reached. The ability of the common man to ignore 
promotional appeals that do not interest him is too well known to justify any 
confidence in such methods. 


Ip some cases the effectiveness of a campaign of information can be inferred 
from indexes of public activity coincident with it. If the Department of Agricul- 
ture were to receive an unusual number of requests for one of its pamphlets which 
it had been featuring in its radio and newspaper publicity it might reasonably 
regard this response as a measure of the success of the particular informational 
effort. However, many of the informational programs undertaken by the federal 
agencies are very complicated, large-scale enterprises and do not lend themselves 
to easy evaluation. An agency which has sponsored a major broadside of informa- 
tion such as was involved in the War Bond campaigns requires a detailed tech- 
nique of evaluation which goes far beyond the informal methods of appraisal 
in common use. 


In situations of this kind interview surveys have proved to be a very useful 
instrument. The Treasury Department was able, for example, to find out within 
a few weeks after each bond drive how many people had known the drive was on, 
how many knew the individual and community quotas which had been set, how 
many knew the slogans which had been featured, how many knew the relation of 
bonds to inflation control, and other related items. It was able to determine which 
parts of the population had been most effectively reached and which were least 
well informed. Most important of all, it was able to assess the influence of these 
items of information on the individual’s inclination to buy bonds. 
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In most surveys of informational policy, the objective is not only to find out 
if the campaign has been successful but also to learn from the reactions of the 
public how to shape subsequent campaigns in the most effective manner. This 
was true in the case of the surveys of the War Bond Drives and it was also a fea- 
ture of several studies of the program of nutritional education which the War 
Food Administration carried on during the war. It is not sufficient, for example, 
to show by means of a survey that campaigns on information regarding nutrition 
have reached only a small minority of the population. A full answer to the prob- 
lem of educating the public regarding food requires an understanding of why the 
public has not responded to programs which have been tried and the determina- 
tion of which of the techniques used hold the most promise of subsequent ex- 
ploitation. 


Measuring Public Information as a Basis for Program Planning 


Survey research on public information has also proved valuable as an aid in 
the development of new programs on which federal administrators propose to 
take action. Agencies which plan to introduce new policies for which the public 
has not been prepared may very seriously misjudge public response to the program 
if they cannot correctly estimate the degree to which the public understands the 
problems involved. 


At the present time, for example, it is evident from survey research that 
relatively few farmers in this country are aware of the implications of the so- 
called Steagall Amendment which guarantees a certain price on most important 
farm products for two years following the end of the war. The Department of 
Agriculture cannot successfully carry forward any program which requires an 
understanding of the Steagall Amendment until further informational work has 
been done. In the same manner, the Bureau of Internal Revenue must consider 
the extent to which the tax-paying pub! . will understand the language and con- 
cepts it proposes to use in its tax return blanks. In order to adapt its tax return 
forms to the informational level of those who will be required to use them, the 
Bureau commonly conducts small-scale surveys designed to reveal whatever mis- 
understanding and confusion its proposed forms may create. 


Information of this kind defines the boundaries of public understanding. 
It enables the administrator to estimate realistically the ability of the public to 
deal with the program he has in mind. 
How Does the Public Feel About Actions the Government has Taken? 


It is often a most difficult problem for the federal administrator to find out 
why his efforts have been a success or failure so far as public cooperation is con- 
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cerned. It is one thing to discover that the public has not responded to a program 
in the way the administrator hoped. It is often quite another to achieve a 
thorough explanation of why the public has responded as it has. Such an explana- 
tion requires an understanding of how the public views the program, what its 
attitudes toward the program are. 


A program which has been conceived with the most innocent intentions may 
be regarded by the people it affects as an outrageous infringement of their per- 
sonal rights. A few years ago one of the federal agencies which exercises control 
of lands held in public domain found itself confronted with a series of destructive 
acts against public property in one of its local districts. Although several offenders 
were brought to trial and convicted, the situation was not improved. This agency 
finally discovered through use of survey procedures that certain of its directives 
had restricted privileges which local people had grown to regard as their right. 
Resentment of this restriction was expressed in acts of sabotage against the prop- 
erty controlled by the agency. Once these facts were understood, the agency was 
able to modify its administrative procedures and the problem was quickly solved. 


Many other examples might be given of the importance of surveys of this 
type, designed to answer questions of why. Throughout the war period the 
Treasury Department needed to know why some people were cashing their bonds. 
Was it because of fear that the bonds would not be redeemed at full value, be- 
cause social pressure had compelled people to buy more bonds than they could 
conveniently hold, or because of unforeseen emergencies which required un- 
usually large expenditures? In a similar fashion the War Food Administration 
needed to know the factors which stimulated or restrained the farmer’s plans for 
production. Did he respond to patriotic appeals for wartime effort, or was he 
mainly interested in higher prices? The Office of Price Administration needed 
to know the way in which consumers viewed its activities in order to understand 
fully their response to the rationing and price control regulations. 


Through analysis of the public’s appraisal of their programs, federal admin- 
istrators have been able to evaluate the flaws and strong points of their operations. 
There are, of course, many factors besides public acceptance which determine the 
success of a government program. But, as pointed out in the preceding article, 
there are many government programs which cannot succeed without public ap- 
proval and cooperation. The administrator is in a very much better position to 
achieve public support if he has dependable information on the way his policies 
are being received by the public. In the absence of such information he must 
proceed in virtual blindness, dependent on rumors and conjecture which may 
bear little relation to the facts. 
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How Will the Public Feel About New Government Action 
That is Being Planned? 


Ordinarily the federal administrator has some latitude in determining the 
type of program to be used to meet his special problem. He nearly always has 
some choice in the procedure he will follow and can consider public response as 
one of the factors determining his decision. By means of the survey technique 
he can obtain a measure of public attitudes toward the alternative programs which 
he is considering before they are actually put into operation. This does not mean, 
of course, that he will base his operating policy solely on the prospect of public 
approval. The program which is most feasible from the standpoint of adminis- 
tration may be the least acceptable to the public. In this case the survey's chief 
value is to make clear why the proposed program is unpopular. On the basis of 
this information the administrator may well be able to aiter those aspects of the 
program which will meet greatest resistance, and yet to retain the essential char- 
acteristics of its operation. 


A striking example of this use of survey procedures is found in the point 
system of discharge which the War Department has followed in reducing the size 
of the Army. One of the major considerations taken into account by the War 
Department in determining the relative weight to be assigned to the various 
factors for which points were given was the preferences of the soldiers them- 
selves, as expressed through the medium of surveys. 


The decision made by the Treasury Department during the war period to 
reduce the time delay in the redemption of War Bonds is a further illustration 
of this application of interview surveys. In this case the Treasury knew from 
survey findings that a good many prospective bond buyers said they would buy 
more bonds if they were more liquid; these people were keeping their savings in 
other forms because they wanted to be able to meet any unforeseen emergency. 
The Treasury also had good reason to believe froin its survey data that most bond 
redemptions were not motivated by capricious impulses to spend but by unusual 
demands such as income tax payments or hospitalization expenses. Its conclusion 
from its investigations of public attitudes was that a simplified procedure of 
redemption would satisfy the needs of some prospective buyers for a reasonably 
liquid form of saving and would not, on the other hand, increase the rate of 
redemption by the total population of bond owners. 


There is, of course, some danger in inferring the eventual public acceptance 
of a program from public attitudes toward a proposal which is not yet in opera- 
tion. The public’s actual experience with the program may be very different from 
its expectation. However, many projected programs are adaptations of procedures 
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which have been used before and with which the public has some familiarity. 
Even in those cases in which the program involves operations with which the 
public is entirely unfamiliar, detailed interviewing can present its major features 
in a realistic way, and a preview of the kinds of arguments which will be brought 
against the program and their relative importance can be obtained. 


What is the Public Doing? 


Many federal programs depend dire-“'” on a continuing reporting of con- 
temporary public activity. The Treasury Department needs to know, for example, 
how many bonds are being bought each month and how many are being cashed. 
During the war the OPA had to have continuing reports on many questions re- 
garding wholesaler, retailer, and consumer activity. The Department of Labor 
must know the size of the labor force and the extent and disposition of unem- 
ployment. Dozens of federal offices are dependent on current information of 
this type. 


There are various ways in which this vital information is obtained. A great 
-deal of it comes from the records of the agency’s own activity, as in the case of 
the Treasury Department in the accounting of bond sales and redemptions. Much 
of this information comes through business and financial records which are avail- 
able to federal offices. For some of this information, however, there is no institu- 
tional source. It can only be obtained by going directly to the people involved. 
Numerous examples of this use of survey research might be cited. A monthly 
report on the labor force is prepared by the Bureau of the Census on the basis 
of interview surveys. The Department of Agriculture conducts a quarterly survey 
of the activities of the nation’s farmers. Many smaller scale surveys are done 
continually to provide government agencies with the data on contemporary public 
activity which they must have for efficient operation. 


What is the Public Planning to Do in the Future? 


Interview surveys have achieved their greatest publicity through their ability 
to predict correctly how the electorate will vote. In predicting the 1944 presi- 
dential vote, all of the major polling agencies named the eventual winner and 
none was more than two percentage points away from the popular vote. 


However, the prediction of voting behavior is only the most spectacular of a 
variety of ways in which survey research can be used to predict what people are 
going to do. Numerous inquiries into the future plans of the populace have been 
conducted during the last few years, dealing mainly with expectations of post-war 
activities. The War Department, for example, has released a series of reports on 
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the intentions of soldiers regarding postwar employment, migration, and school- 
ing. The Federal Reserve Board has sponsored a large scale study of public ex- 
pectations regarding the use of accumulated liquid assets. A number of survey 
studies of post-war prospects have been done by local municipalities during the 
last two or three years; some of these have received aid from government survey- 
ing agencies. 


The assessment of public intentions through the use of survey research is in 
some respects a rather new development and is regarded cautiously by many 
administrators who are accustomed to predicting public behavior on the basis of 
more conventional principles of past experience or economic law. The prospect 
of going directly to John Doe’s door and asking him what his plans are seems 
unfamiliar and untried. The future of this phase of survey research depends, of 
course, on its ability to predict public activity more effectively than can be done 
by more traditional methods. 


Are There Public Needs on Which the Government 
Should Take Action? 


The public is in a constant state of change, in both its achievements and its 
aspirations. Depression and war upset countless well-ordered modes of living 
and introduced new cultural stresses and inadequacies. It is obviously important 
that the Federal Government should be informed about these developing strains 
in the institutional structure so that action can be taken to ease the transition to 
new arrangements. 


The Federal Government, in both its legislative and executive branches, has 
traditionally been more inclined to take action after public need has reached a 
state of desperation than to anticipate difficulty and to minimize it by programs 
devised to meet the situation. During the last few years, however, there has de- 
veloped a somewhat different attitude toward administrative planning both in 
Congress and in the government agencies. Many federal administrators now 
make a systematic effort to anticipate developing inadequacies in the operation of 
their programs. An administrator who suspects from his knowledge of general 
social and economic changes that the needs of the people affected by his program 
may be changing finds it possible to obtain through survey prc_edures a report 
of the situation of the people and an indication of the kinds of administrative 
action which are needed. 


An example of the usefulness of research of this type is found in the prob- 
lem of the returned veteran. Since it was obvious before large-scale discharges 
from the armed forces were begun that the return to civilian life would abruptly 
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put millions of veterans into a situation with many psychological problems, it was 
thought advisable to examine the typical experiences which veterans already dis- 
charged had encountered and how they had adjusted to them. From this informa- 
tion the responsible government officials were able to devise a program of educa- 
tion for army personnel yet to be discharged based on a realistic appreciation of 
the difficulties which these veterans would meet. 


The Federal Communications Commission has recently conducted a nation- 
wide survey to determine the quality of radio reception in the rural areas of the 
country. Since it is the responsibility of the Commission to regulate the licensing 
of radio stations in such a way that “the public convenience, interest, or necessity 
will be served,” it felt it necessary to ascertain from those listeners who are most 
remote from the broadcasting centers whether the radio service they receive is 
adequate. The findings of the survey have become an important consideration in 
the Commission’s deliberations regarding ways and means of insuring more 
equitable radio service for all the people. 


During the war the Office of Civilian Requirements of the War Production 
Board carried out a series of surveys intended to assess the public need for various 


‘consumer goods which were then in a very short supply. Their findings provided 


the basis for the allocation of the restricted supply of material and manpower to 
the production of those articles for which there was the greatest need. 
* * * 


Ordinarily, of course, the questions which the federal administrator wants 
answered do not fall exclusively within any one of these areas of inquiry. Much 
more commonly his objectives require a broad investigation of attitudes, under- 
standing, and activities. The study he wants done may include several or all of 
the kinds of information enumerated in the preceding pages. Let us suppose, for 
example, that the director of one of the federal home loan agencies wanted a full 
assessment of the operation of his program. He would at least want answers to 
the following questions: 


How well do the borrowers understand the terms of their contracts? What 
parts of the agreement are not well understood ? 

To what extent are the borrowers following the terms on which their loans were 
given? In making improvements on their house and land? In budgeting income and 
planning expenditures? In meeting the payments due on the loan? 

What are the attitudes of the borrowers toward the terms of their contracts? 
Do they think of them as a help or a nuisance? 

What suggestions do the borrowers have for ways in which the lending program 
could be improved ? 

What are the attitudes of the borrowers toward the representatives of the lend- 
ing agency? What criticisms do they have of the way these people do their job? 

How would the borrowers feel about proposed changes in the operation of the 
program? What reasons do they give for approval or disapproval ? 

What differences in response to these questions exist between borrowers of 
different regions, of different ages, of different educational levels? 


21 











It must be apparent that a survey designed to examine a complicated program 
of this kind bears little resemblance to the polls of public opinion to which news- 
paper readers are accustomed. Most government agencies are concerned with 
problem analysis rather than polling and require much more highly developed 
procedures than are used in most polls. 


Conclusion 


Interview surveys have been widely used by the various agencies of the 
Federal Government during the past five years. Surveys of public attitudes, public 
information, and public activities have provided the data on which many impor- 
tant administrative decisions have been based. 


At the present time, the future prospects of survey research in government 
are rather uncertain. Interest in this type of research as an instrument of adminis- 
tration increased greatly during the war and will undoubtedly continue to be high 
among those administrators who understand its usefulness. However, the end 
of the war period will bring pressures for widespread reductions in government 
expenditures and it seems probable, from historical precedent, that research will 
be one of the first government activities to be labelled ‘“‘non-essential’”’. It remains 
to be seen to what extent the executive and legislative branches of the Government 
will feel they can afford to return to the unreliable and inefficient methods of 
gathering information which were in use before the advent of interview surveys. 














Public Opinion Polls and Democratic Leadership 


DorwWIN CARTWRIGHT 


The great potentialities of sample surveys in serving both the legislative and 
administrative branches of the Government are now well established. Through 
an extremely rapid growth of the science of sampling it has become possible to 
obtain relatively quickly and with moderate expense an accurate miniature of the 
total population. Thus, in a sense, it is possible to convene the entire population 
of the country and to get an expression of opinions and desires with an ease and 
efficiency never dreamed of by the framers of the Constitution. 


The Founding Fathers, not knowing of the science of sampling, could, of 
course, make no provision for public opinion research in the democratic process 
of government. Their method for keeping control in the hands of the people was 
that of assuring free elections in which /eaders were chosen to represent the 
people. While a voter might have some notion of how his representative would 
. vote on specific issues, he would have to take his chances that his spokesman 
would actually follow his wishes. The democratic control, and the very essence 
of democracy, lay in the fact that ‘elected leaders would have to stand before the 
people periodically for re-election. In the years that have followed, this control 
has proved to be quite limited; on many issues the will of the people has been 
but weakly exercised. 


In the executive branch of the government democratic control was made even 
more indirect. Activities and policies of the administrative agencies were to be 
determined by elected representatives, the President and the Congress; not by 
popular vote. Had the founders of our government foreseen the tremendous 
growth of bureaucracy which has taken place since their day and had they been 
able to imagine the intimate way in which the executive agencies would come to 
touch the lives of all our citizens, it is possible that they would have devised some 
more direct method of control over this branch of government. Had they known 
of public opinion research, they might have made provision in the Constitution 
for its regular use. 


Pressure groups constitute a most impressive symptom of the illness of our 
democratic functioning. Much has been written about them but, like ghosts, one 
has to see them to believe in them. A visit to Washington on any day chosen at 
random will reveal that not only the offices of Congressmen, but also the corridors 
of the administrative buildings, are frequented by special pleaders claiming to 
represent the wishes of one group or another. Techniques of lobbying have be- 
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come so highly developed that persons with a professional reputation in the field 
are avidly sought by groups able to pay well to have their wishes felt in Washing- 
ton. 


Against such a barrage the conscientious public servant has little defense. 
What can he do to determine the true wishes of the public? He can read edi- 
torials (known to be unrepresentative) ; he can have his mail tabulated (hardly 
more trustworthy) ; he can take a field trip and search out the opinions of “‘typ- 
ical” people (but don’t ask him to prove they are typical) ; he can even follow 
the procedure employed by the O.P.A. and bring into Washington an “average 
housewife”. Surely, the citizen who doesn’t belong to a strong organization with 
a lobby in Washington or who doesn’t know a Congressman’s wife has little 
chance of influencing governmental policies and procedures except through rare 
visits to the voting booth. 


Looking at the government through the eyes of those who shape policies, 
there is need to keep in touch with the public for yet another reason. It is not 
enough to know whether or not the public is in favor of some broadly defined 
program; let us say, inflation control. Even though its objectives receive over- 
whelming approval, a program may fail because its detailed operation does not 
correctly take into account the “human element” inherent in any public action. 
There is every indication that the public fully supported a government program 
of keeping prices down during the war, and yet a war bond program based solely 
upon an anti-inflationary appeal would not have sold enough bonds because the 
public did not generally see the connection between buying bonds and the prices 
of commodities. Nor would any program composed solely of advertising have 
done the trick. Intensive research for the Treasury showed that large-scale organ- 
ization for face-to-face selling was required in spite of the fact that the War Bond 
program was one of the most popular of the Government's wartime activities. 


It is clear, of course, that an experienced administrator will know a good 
deal about the “human element” in public administration even without the aid 
of public opinion research. The Treasury, for example, by enlisting the full 
support of the advertising industry and the aid of its best men in selling bonds, 
tapped a vast wealth of such experience. And yet these experts insisted that 
facilities be made available for keeping them closely in touch with the thinking 
of the public. They knew all too well that hunches and intuitive principles could 
lead them far astray. They had found that business survival demanded the type 
of knowledge provided by “market research”. 


Considering only briefly, then, some of the barriers to effective democratic 
government under modern conditions, the potential contributions of sample 
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surveys appear to be truly great. Isn’t it self-evident that legislators and adminis- 
trators should employ public opinion research at every opportunity? Yet, even if 
this be granted, certain cautions need to be exercised. Consideration of the proper 
function of leadership in a democracy and facts about the actual present-day 
operation of public opinion research should restrain a bit the enthusiasm of the 
extremists. Let us examine a little more closely what the ideal relation between 
the people and t‘eir leaders might be. 


What do we want of our leaders in a democracy? What should they do 
for us? Certainly they should represent the will of the majority; we all agree to 
that. But they should do more. We want men who will lead as well as follow 
the public will; we want experts who know more about the subject than anyone 
else; we want men of vision who can invent new solutions to our problems. No 
doubt a poll of the American people would reveal support of each of these 
propositions. If the public really accepts these functions of the leader, however, 
it then follows that leaders should by no means always abide by the results of 
the polls. There are certain circumstances under which they are obligated to dis- 


regard them. 


Public Information and the Expert 


Although the educational level of this country is high and media of informa- 
tion reach every segment of the population daily, popular knowledge of public 
affairs is meager. Many examples could be cited showing how little people know 
about these matters. For instance, at the height of discussion of the Little Steel 
Formula, April, 1945, only 52 percent of those approached by Gallup inter- 
viewers asserted that they had heard of it, and only 30 percent could give a 
reasonably accurate indication of what it was intended to do. It has been found 
on several occasions that less than half of the population can give correctly such 
items of information as the number of years for which a member of the House of 
Representatives is elected. Perhaps more revealing is the finding of the Fortune 
Poll in August, 1944, that 46 percent of the population thought that John L. 
Lewis was president of the C.I.O. while nearly one percent mentioned Eric 
Johnston and another Beardsley Ruml. In May, 1945, Gallup reported that only 
41 percent of the adult population knew what the T.V.A. is, and in June of the 
same year he found that only 31 percent knew what the O.W.I. did. Even if 
public awareness were increased ten-fold, it is doubtful that the “average man” 
would have sufficient knowledge of most governmental problems to be able to 
make intelligent decisions without special information and study. 


One may ask what special significance for polling this lack of information 
has. Isn’t it simply a problem of the democratic form of government, and doesn’t 
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it imply only that public education needs extending? Probably it is not that 
simple. The kind of government established by our Constitution did not en- 
vision going to the public for the solution of the thousands of problems which 
arise each year. It was intended that these problems would be solved by leaders 
and experts selected by the people. Public opinion polls which go to the people 
and ask them to choose among specific proposals for the solution of these detailed 
problems actually force many people to make pronouncements upon matters about 
which they know little. To expect government leaders to follow these choices as 
a mandate from the people is to substitute mediocrity for expertness. 


Let us consider a specific example. In a survey conducted in April, 1944, 
by the Division of Program Surveys of the Department of Agriculture, less than 
60 percent of the population could give an explanation of what makes prices rise 
in time of war that showed any substantial understanding of the causes accepted 
by economists. Since it was clear that the public wanted prices held down, gov- 
ernment officials had the responsibility to try to keep them under control, but it 
is doubtful that officials could have best accomplished this goal by asking the 
public to choose the details of an inflation control program. Had they conducted 
a poll for this purpose and followed the advice of the public rather than that of 
economics experts, it is likely that they would not have been re-elected to office, 
for the public was certainly more interested in keeping prices down than in seeing 
some one vaguely conceived program of price control put into operation. 


It seems to be equally inappropriate to ask the public what the income tax 
rate should be for the different income brackets. The fact that polling organiza- 
tions have repeatedly found that people choose tax rates for the upper brackets 
far below present levels carries no obligation to Congress to lower these rates. 
It simply indicates how hard it is for people to imagine anyone else paying more 
in taxes than they themselves earn. Although the case is not so clear, the question 
of the sales tax is of the same kind. Economists agree that it is ‘regressive’, 
placing a heavier burden on those least able to pay. That the little man does not 
recognize this fact and is for some reason enticed into supporting the tax in a 
poll seems poor reason to impose it upon him. Leaders who represent the little 
man and who know the facts about the sales tax will be unimpressed by a survey 
that has forced the unsophisticated laymen to make a pronouncement on economic 
policy. 

A serious defect in the average public opinion poll is that by posing ‘‘issues” 
to a person it is possible to get him to select among proposals about which he has 
little knowledge and little conviction. Under these circumstances, ‘“‘overwhelm- 
ing” support of a proposal, in the sense that a large number of people select it, 
does not necessarily mean at all that the public is intensely anxious to have the 
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proposal adopted. It may mean that most people know little or nothing about 
the details of the proposal but see it as being related to some value which they 
do accept. For example, during the war most people felt that winning the war 
was the most important consideration that should apply to any public question. 
Holding this belief and not knowing much about the details of government 
operation and economic functioning, what could they do but approve almost any 
specific proposal presented to them as a means of winning the war? 


From analysis of many questions put to the American people during the war, 
Gallup has concluded that the people were, in general, ahead of the Congress in 
approval of vigorous and even restrictive measures for winning the war. That 
the people were willing to make necessary sacrifices there can be no doubt. But 
it is extremely doubtful that people actually felt a need for all the specific pro- 
posals they endorsed when put to them by enterprising pollsters. Consider the 
following questions asked by the Gallup Poll: 

“Do you approve or disapprove of the proposal to draft nurses to serve with 
the Army and Navy?” (February, 1945) 


“Government and Army officials say it is absolutely necessary to have more 
men to work in munitions plants and war industries. Do you think Congress should 
pass a law to permit local draft boards to draft civilians between ages of 21 and 45 
who are able to work for these industries?” (February, 1945) 


“Do you think Congress should reduce income taxes this year, or should this 
wait until after Japan is defeated?” (June, 1945) 


In each instance a clear majority favored the “tougher” policy; in two of them 
approximately three-fourths of the population agreed. Were people angered 
that their ‘‘will” had been ignored when Congress failed to follow all of these 


and other proposals tested by the pollsters? There is very little indication that 
they were. 


The crucial fact is that a specific proposal about which the public has little 
information will be judged in terms of the values to which it seems to be related. 
Mutually contradictory proposals will then be approved if they are seen as related 
to approved values and if the contradiction is not made too obvious in the ques- 
tionnaire. Similarly, when information is meager, minor variations in the word- 
ing of the question can produce widely different answers. Surely, an administra- 
tor can be seriously misled if he follows uncritically the majority opinion regis- 
tered in a poll without knowing the level of information and strength of con- 
viction associated with that opinion. 


In the area of international affairs this problem is particularly acute. Public 
thinking about these matters is limited, but values and stereotypes abound. ‘‘Over- 
whelming” support of proposals not clearly conceived nor firmly held can be 
readily found in a public opinion poll. The desire for peace is so strong that 
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specific proposals presented as contributing to the attainment of peace have a 
good chance of being approved if the pollster on the doorstep insists on an 
answer. Compare the following two questions and the answers to each. 

“After the war, would you like to see the United States join some kind of world 


organization, or would you like to see us stay out?” (National Opinion Research 
Center, January, 1945) 





CRT eo anne ee 64% 
PRG NRE Sacco ssecuvssh cnusstuctochsopvsteoseses 26% 
Undecided - wee 10% 


“Do you think the United States should join a world organization with police 
power to maintain world peace?” (American Institute of Public Opinion, April, 








1945) 
Yes ae 81% 
BU esse esasah spnceonosnancs 11% 
DD IND oss ssscsscasvcrsessvacesssssosens 8% 


The difference between 64 and 81 percent is considerable and could have strik- 
ingly different practical consequences. Unless the busy administrator realizes the 
magical quality of the phrase “to maintain world peace”, he might be misguided 
through reliance upon the findings of a given poll. 


A particularly dangerous error is sometimes made in the use of public 
opinion findings—especially in the area of thinking about international affairs. 
Questions like the ones just mentioned refer to a vague area of thought perhaps 
best labelled as “friendly cooperation for world peace”. While it is important to 
recognize that the American public has swung sharply since the beginning of the 
war toward accepting this general philosophy, it would be a serious mistake to 
assume that the public has much understanding or ready support for proposals 
for world organization which do not automatically appear to be related to these 
values. Take the loan to Britain for instance. Those who had been impressed by 
the new era of world cooperation as heralded by the polls were rudely awakened 
when the polls got around to the British loan. Although roughly two-thirds of 
the population favor American participation in a world organization, only one- 
third approve of the loan to Britain. While the public will approve many specific 
proposals which are seen by them as being a part of a program of friendly coop- 
eration, they will reject others which experts feel to be equally essential to such a 
program simply because these proposals call into play other sets of values. Public 
opinion research which doesn’t get at the full story can seriously mislead those 
who try to follow it. 


In summary, then, it is clear that the public does not have sufficient knowl- 
edge about most governmental problems to pass intelligent judgment upon them. 
This is not at all to say that the public is negligent of its duties in a democracy. 
Modern government has become so extensive and so complicated that even experts 
must specialize on certain types of problems. The danger of applying public 
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opinion polls to these areas lies in the fact that people can be induced to express 
an opinion in a poll on matters about which they know little. They often do so 
without any intention that their opinions should result in government action. By 
asking policymakers to follow the results of such polls we are substituting poorer 
knowledge for better. Only when a problem has become subject to extensive 
public discussion and issues clearly drawn should a government leader seriously 
consider polling results as a mandate. Even then, he may feel compelled to 
follow his own judgment of what is best for the country. 


Technical problems associated with polling about matters of little public 
knowledge strengthen this conclusion. Questions which give fixed alternatives to 
the respondent may badly mislead a policymaker in that large percentages of 
approval need not indicate at all conviction of intensity of feeling. Moreover, 
they tend to conceal lack of knowledge and understanding. It is doubtful that 
such questions by themselves should ever be taken by a government leader as a 
demand for action. Fortunately, techniques have been developed which go far 
toward solving these difficulties. The use of “open” interviews and of new tech- 
niques for measuring intensity of feeling can minimize greatly the dangers in- 
_ volved in traditional polling. Research on public thinking can contribute greatly 
to the democratic process, but failure to recognize and overcome these technical 
problems can vitiate entirely the values of such research. 


Leadership and Invention 


Social progress, like technological progress, depends upon the widespread 
adoption of new procedures which have been developed by a relatively small 
number of people. As much as we might wish it, the average man is neither an 
inventor nor an innovator. It is hard for him to imagine conditions or states of 
affairs which differ much from the familiar. The public stereotype of the inventor 
as something of an eccentric reflects this fact that the creative thinker is different 
from the average run of man. If you ask the average man to solve some problem, 
he tends to rely upon familiar methods; his thinking is confined to customary 
ways of viewing the matter. Were this not the case, each of us could be inventors. 


In another way the inventor differs from the rest of us. He is a restless soul 
who is never satisfied with the present condition of things. He is constantly 
looking for improvements, for better procedures. But only when our needs are 
seriously thwarted do we common mortals search out new practices. If the horse 
will get us where we want to go, we are happy—until the automobile leaves us 
in its dust. We are quick to adopt a new and better creation after we have seen 
it work, but we are slow to perceive its need and to invent it. 
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Applied to government this means that our institutional procedures should 
provide adequately for social inventiveness on the part of specialists rather than 
rely upon the masses for social innovations. We recognize that the truly great 
leader is the man who can see the possibilities for improvement in an imagined 
new state of affairs. The successful leader is the man who can get the public to 
adopt these new social inventions. To the extent that we desire social progress, 
we must seek leaders who can surpass the rest ” us in finding new and better 
solutions to our social problems. 


There is evident at the present time a tendency to make use of public opinion 
research in a way that would destroy this function of leadership. In the guise of 
being democratic and of giving the average man a greater voice in social affairs, 
public opinion research can be used to impede progress through misplacing the 
function of invention in our society. By asking the public to invent solutions to 
social problems and by interpreting the absence of new solutions as a desire for 
the status quo, public opinion polls are sometimes employed to bring pressure to 
bear against innovation and change. 


Consider an example in government. From the establishment of the federal 
income tax in 1913 until 1943 our system of tax collection was complicated, 
confusing, and inconvenient to the person with moderate income. Yet, com- 
plaints about the method of collection were few just because that was the way 
things were done. Suppose, then, in 1941 a poll had been conducted in which 
taxpayers were asked whether they could think of any better way of collecting 
taxes. There is little reason to believe that suggestions would have been num- 
erous, and the Treasury Department might well have decided that nothing new 
was needed. In 1943, however, just as soon as a withholding tax was inaugurated 
in the form of the Victory Tax, there was overwhelming approval in the polls of 
having all income taxes deducted from pay. From this reaction, it is clear that it 
would have been wrong to assume from the poll in 1941 that the public was 
wedded to the status quo in tax collection. 


The situation is basically the same in any poll which asks the public to sug- 
gest solutions to social problems. What should be done to solve the unemploy- 
ment problem in the future? What should our policy be toward Germany? What 
should be done about a probable surplus of wheat production within a few years? 
Unless some new solution to such problems has been widely publicized, people 
will tend to think only of techniques which they have already experienced. Take 
the question of surplus wheat as an example. In public opinion interviews wheat 
farmers propose solutions to this problem which constitute essentially the old 
A.A.A. program that social inventor Wallace got farmers to adopt during the 
depression. A public opinion survey in 1929, however, would not have yielded 
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many suggestions for this type of program. Only after farmers had become 
familiar with its workings, could they apply its approach to future problems. But 
now the situation has changed again, and many agricultural economists believe 
that the A.A.A. type of program is no longer appropriate, that it can be viewed 
only as emergency patchwork, and that a basically new orientation is needed. If 
government leaders take the findings of the current polls as evidence that the 
public demands continuation of the A.A.A. kind of program, the same creative 
influences which originally produced the A.A.A. will be hampered in bringing 
about further improvement in our agricultural economy. 


The use of public opinion research to justify the status guo in any area of 
public life deserves the same criticism. Radio programming provides a partic- 
ularly interesting illustration. People are no more inclined to have suggestions 
for changing their radio service than they are for changing any other part of their 
world. Vested interests which wish to maintain present programming in radio 
can easily collect public opinion data to show how few people have any sugges- 
tions for basic changes in their radio service. Experience has shown, however, 
that new types of programs, once created, may become extremely popular over 


_ might. Had a survey been conducted a few years ago asking for suggested changes 


in radio programs, probably no one would have suggested “‘a new type of pro- 
gtam in which someone asks people questions”. But today quiz programs rate 
high in listener appeal. By the same token it is unjustified to take current polling 
results that over half of the population have no suggestions for improving radio 
service as a reason for satisfaction with present standards of radio programming. 


From quite a different area of life comes a most dramatic example of how 
far wrong one can go in predicting future behavior on the basis of polling ques- 
tions which ask people to construct new situations. The British Institute of Public 
Opinion asked the British public in March of 1945 the following question: 


“Who would you like to see leading the new government after the war?” 

Note that the question was asked before V-E day, but only a few months before 
the actual election. In view of the election results, the answers to the polling 
question would seem almost unbelievable. Yet it must be kept in mind that this 
question is one which asked people to imagine a state of affairs that is quite differ- 
ent from the one in which they found themselves at the time. Accordingly, we 
should expect the results of the poll to reflect wartime thinking more closely than 
postwar behavior. Actually, the polling results did just that. The answers were 
as follows: 


Eis sindiishbinlalalessibpeiiiianies 31% 
ee 20% 
I itesseninsiniicindasuonnnnies 6% 











See eeeerereeeeeeceseesseressseeees 








Morrison 
NEE watinhceineiicianainnsitene 1% 
I sissssnesnaniciusonsvnvncsenics 1% 





Had one interpreted this finding as a strong wish for maintenance of the status 
quo in British political leadership after the war, the subsequent election would 
have come as quite a shock. 


From this discussion it would be incorrect to conclude that public opinion 
research can make no contribution to invention and progress in government. On 
the contrary, it is just this area of public life where it can be of the greatest 
assistance. Properly designed and executed research on public needs can give 
social inventors goals to aim at. As the previous article has indicated, studies of 
the efficiency and effectiveness of existing governmental procedures can give leads 
for further improvements. Sample surveys can be very useful in determining 
whether a program that is sound from an economics point-of-view will be at all 
satisfactory when the “Shuman element”’ is considered. 


Summary 


Modern democratic government needs public opinion research to make it 
function more smoothly and more democratically. The many possible contribu- 
tions of sample surveys have been enumerated here and elsewhere in this issue. 
But unrestrained enthusiasm would be out of order. The promiscuous application 
of public opinion polls to all matters of government would be inadvisable and 
actually dangerous for it would take away from our leaders important functions 
which they alone can adequately perform. As a result of enthusiasm for a new 
and useful device, we must not ask our leaders to give up their proper roles of 
leadership. Polls must not divest them of their duties as experts and inventors. 


It would be a display of shortsightedness, however, if for the reason that 
polls can be misused, we were to fail to use them at all. With care and with 
earnest avoidance of misapplication we can greatly improve our methods of self- 
government through public opinion research. Our choice is not whether to use 
sample surveys or to abandon them. We must use them now that we know how. 
Let us only be sure that we use them properly. 








The Interpretation of Survey Findings 


DANIEL Katz 


The results of scientific research are sometimes regarded as pure objective 
statements of absolute truth devoid of interpretation. But if research findings 
are to have even scientific meaning, they must be interpreted in terms of the 
conditions and assumptions of the investigation of which they are the outcome. 
Merely because an interpretation is not expressly stated in a report does not mean 
that no inferences are drawn from the presentation of results. In fact the failure 
to make explicit the assumptions and conditions of the research investigation 
creates the impression of a finding more universal than the facts justify. Scientific 
findings must always be considered both in the framework of the particular re- 
search study and in relation to similar studies in the same field of knowledge. 


The inevitability of interpretation of scientific results imposes the task of 
criteria and precautions for the evaluation of data. And the role of interpreta- 
tion takes on new and gigantic proportions in public opinion research. When 
we study the world of physical facts, our own values do not intrude to color the 
picture as much as when we make human values the core of our research. 


Most polling of public opinion is done by commercial agencies which con- 
duct surveys either for specially interested clients or for newspapers and maga- 
zines. Most of the widely publicized poll findings are news services sold to news- 
papers or magazines. These surveys depend for their existence upon producing 
news-worthy releases and hence there is some pressure in the direction of the 
politically-interesting question of the moment rather than toward thorough-going 
and fundamental research. It must be remembered too that a commercial organ- 
ization is in business to make money and that the more comprehensive and scien- 
tific the study, the higher the cost. It is also true that the poll operator as a busi- 
ness executive is more likely to have the values of his group than the values of 
the more disinterested academician. That the polling agencies should have their 
limitations as research organizations is, therefore, to be expected. The surprising 
fact is that they have yielded so much valuable data both for practical and for 
scientific purposes. 


Even surveys not made for popular consumption are subject to similar 
influences in that they are conducted for a practical purpose, sometimes by the 
Government and sometimes by a commercial concern. High research standards 
can and have been achieved under these conditions, but since so much of the 
work in public opinion and social science is “applied” rather than “‘pure’’ re- 
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search, the interpretation of findings becomes a crucial problem. The following 
cautions are suggested, therefore, both to guard against common misinterpreta- 
tions, about which the “consumer” of facts must be aware, and to point to better 
research procedures. 


The Confusion Between Problems of Fact and 
Problems of Opinion About Fact 


The discovery that people’s attitudes and opinions can be readily ascertained 
has led to an indiscriminate attempt to get answers to problems of fact by count- 
ing noses. Now when we want to know about people’s fears, hopes, aspirations, 
satisfactions and dissatisfactions, wishes, beliefs and preferences we naturally 
and justifiably conduct attitude surveys. But when we want to know facts that 
lie outside of people’s subjective feelings, we study the problem directly. It is of 
some importance, for example, to know how people feel about the treatment of 
cancer, but public attitudes regarding the efficacy of various treatments cannot be 
taken as a measure of the actual effectiveness of these treatments. 


At this simple level, it is not hard to separate the two problems. What often 
happens, however, in public opinion research is this: an attempt is made by an 
interested group to develop a public issue in the press. The real problem is one 
for factual study and cannot be solved by public debate. But public opinion polls 
seize upon the issue, conduct a survey, announce how the American people think 
about the matter and so join the debate. 


For example, during the war the problem of absenteeism in war industry was 
presented in some papers as a crusade against the malingering war worker. The 
real problem was factual in nature: How much absenteeism was there, what were 
its causes, how could they be treated to reduce absenteeism? Factual studies were 
made at this time by industry itself, by the War Manpower Commission, and by 
OWI. The factual researches indicated that the incidence of absenteeism was 
exaggerated, that it was due in large measure to the mushrooming of war industry 
with increased employment of working wives and mothers. The American Insti- 
tute of Public Opinion, however, disregarding these facts about absenteeism, 
polled the American public on the question: 


“What do you think should be done with workers in war factories who are 
regularly absent from work without a good excuse?” 


The Gallup release to the press reported the answers to this question under 
a headline stating that the great majority of the American people supported 
Rickenbacker in his campaign against absenteeism. Both the phrasing of the 
question and the presentation of the results assumed malingering on the part of 
war workers—the very problem that called for factual investigation. 
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Aside from the fact that the American Institute study in its very nature 
could contribute no evidence on the problem of absenteeism, there is an additional 
misinterpretation in this particular press release. It assumed that because people 
are opposed to inexcusable and habitual evil doers, that they were therefore in 
favor of the Rickenbacker crusade. Gallup treated the question of what should 
be done with war workers regularly absent from work without good excuse as if 
it had read, “Do you think that Rickenbacker is correct in assuming that many 
war workers are regularly absent from work without good excuse?”. 


The main point that needs emphasis is the need of recognizing the difference 
between people’s judgments and the objective facts. There is some danger in 
substituting opinion-study for factual investigation since it is often easier to poll 
the man in the street than to conduct a direct and thorough study of the problem. 
In interpreting survey findings the first caution is not to make people’s opinions 
about objective facts synonymous with the facts themselves. This is not to mini- 
mize the importance of people's beliefs and values. The ideas people accept may 
be of great significance for social action. But it is necessary to distinguish between 
what people think are the facts and what the facts actually are. 


The Failure to Use Relevant Samples 


Since public opinion polls grew out of forecasting elections, they tend to 
emphasize the reactions of a cross-section of the nation in most of their studies. 
In many problems, however, a national cross-section is not the most relevant 
sample for the hypotheses under investigation. An attitude study of worker 
morale, for example, needs to be directed at a sample of workers. During the 
war governmental studies of problems relating to the war effort became more 
meaningful through the use of special but relevant samples. In checking on 
popular reaction to the administrative functioning of rationing and price control 
it was more significant to talk to the person who did the family shopping than to 
the non-shopper. In getting at reasons for cashing Victory bonds it was important 
to interview a sample of people who were redeeming their bonds. 


Though it is not always practical to set up a special sample for a new study, 
it is possible to ask the factual questions which will permit the respondents to be 
divided into the people with direct experience and knowledge and the people not 
at all involved. Most surveys could be more adequately interpreted if they in- 
cluded in addition to standard factual data such as age, occupation, education 
and urban-rural status the specific experiential material related to the problem 
under investigation. When a study is directed at race relations, questions should 
be asked at the end of the interview about the kind of contacts the respondent has 
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had with members of other racial groups, to find out whether his attitudes are 
based on second-hand experience or on direct contacts under competitive or 
cooperative conditions. 


In a study of white attitudes toward Negroes NORC reported that 6 out of 
10 white persons feel that most Negroes in the United States ace being treated 
fairly. But apparently no attempt was made to get at the basis of these opinions 
in experience, e.g. whether the respondents had ever known any Negroes, whether 
they had worked in industry where Negros had attempted to get jobs, whether 
they had ever witnessed fair or unfair treatment of Negroes. 


The necessity for distinguishing the group with relevant experience is clear 
in some instances, but there is a borderline area in the matter of informed and 
uninformed opinion where there is little, if any, direct experience. Both Gallup 
and Roper often use the filtering question in which respondents are asked if they 
have heard or know about a specific proposal. Sometimes knowledge is spe- 
cifically tested and questions are then asked only of the informed people. This 
is a significant advance but it often is not sufficiently exploited. In October of 
1945 the American Institute used this filtering question: “Have you heard or 
read about Sister Kenny or the Kenny method of treating infantile paralysis?” 
The 52 per cent who answered “Yes” were then asked: “From what you have 
heard or read do you think it is a good way or a bad way to treat the disease?” 
The majority who said they had heard of it thought it was a good method. These 
results have little meaning save as a measure of newspaper treatment of the Kenny 
method, because the kind of knowledge or experience that people have about the 
Kenny procedure was not explored. 


The Failure to Distinguish Between Crystallized and 
Uncrystallized Public Opinion: The Problem of Conviction 


A genuine source of error in interpreting poll returns is to assume that they 
necessarily represent a crystallized public opinion. The issue posed for people 
may not constitute a real problem for them, about which they have strong opin- 
ions. The tendency is to take any question of present or future political interest 
and throw it at the public. The issue may be remote from the every-day lives of 
people; it may have received little public discussion. Hence people have not 
really made up their minds about it. It is not really an issue as far as they are 
concerned.! 


The mistake here stems from a lack of knowledge of the nature of public 
opinion. A mere collection of expressions of opinion from a national sample 


ACE, Katz, D. Three Criteria for Evaluating Polls; Knowledge, Conviction and Sig- 
nificance. Public Opinion Quarterly, 1940, 4, 277-284. 
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does not necessarily constitute public opinion. Because a problem is of practical 
importance or possible political interest does not mean that there is a public 
opinion about it. Public opinion arises only after people have either lived 
through common experiences or have been subject to an educational or propa- 
ganda campaign. Only then are they ready to take a stand upon an issue. 


One indication of the lack of a crystallized public opinion is the percentage 
of “no opinion” answers. As a matter of fact, the percentage of “no opinion” 
answers is often an under-representation, because of the premium in the polling 
procedure on obtaining positive answers. No specific percentage of ‘‘no opinion” 
answers can be set as an absolute index of the lack of crystallization in public 
thinking but this percentage should always be carefully scrutinized. Other criteria 
for evaluating the crystallization of opinion include a study of the immediate 
history of events and of the content of mass media. The following questions, 
selected from the polls during the past year, suggest an uneven level of opinion- 
crystallization which is not always taken into account in the public releases on the 
questions. 


“Should the United States Senate approve the United Nations charter for a 
world organization as adopted at the San Francisco conference?” (July 23, 1945) 








Yes 64% 
NG iccsces i 3% 
No opinion 33%. 





“At the present time when Congress passes a bill to spend money, the President 
cannot veto parts of that bill but must accept it in full or veto it. Do you think this 
should be changed so that the President can veto some items in a bill to spend money 
without vetoing the entire bill?” (November 15, 1945) 








Yes 57% 
No 14% 
No opinion . 29% 





“Do you think the secret of making atomic bombs should be put under the 
control of the new United Nations Security Council, or should the United States 
keep this secret to itself?” (October 27, 1945) 

Put under U.N. control 
U.S. should keep control 
No opinion 

“Should Spain become a member of the United Nations under its present 

government?” (August 15, 1945) 




















. No 76% 
Yes 12% 
No opinion 12% 
“Do you approve or disapprove of removing all price ceilings now?” (October 
23, 1945) 
PDIGRESEOWE! saccscesssssecceasuasssessssconese 72% 
Approve ag 21% 
No opinion 7% 





“In the future, do you think every able-bodied young man should be required 
to take military and naval training for one year?” (November 17, 1945) 








Yes 75% 
No ... 21% 
No opinion 4% 
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The Necessity of a Well-Defined Context for Interpretation 


No single survey finding has much meaning in and of itself. It can be prop- 
erly interpreted only within a well-defined framework, either one that has been 
established by the experimental design of the study or one which the respondent 
has been allowed to establish for himself. Practically, this means that an ade- 
quate study must explore a problem thoroughly ; it must ask not a single question 
but a whole series of questions. It must approach the problem from many angles, 
ask the dependent questions, explore the reasons why, seek the relevant objective 
background material and personal data. In many cases it must be repeated in 
time to give trends. 


The greatest weakness in opinion research to date has been the failure to set 
up thorough studies of the problems under consideration. Poll operators will 
often attempt to cover between 6 and 10 major problems in one questionnaire. A 
number of issues of high ‘attention value” will be summarily treated with a single 
question, Any one of these issues can be appropriately studied only through an 
integrated survey with many interlocking questions with full exploration of the 
significant dimensions of the problem. Without a well-constructed framework 
of hypotheses, survey findings can be interpreted to mean anything and every- 
thing. 

Too often public opinion polls assume that the complexities of a problem — 
can be met by finding some neutral and unambiguous wording of the single ques- 
tion. Somehow such wording is expected to achieve both adequacy and objec- 
tivity of interpretation. Although question wording is important, the assump- 
tion that there is some magic by which a single question can be so well phrased 
with so little suggestive bias that it will furnish adequate and valid answers to a 
complex problem is naive. The emphasis upon the neutral wording of the single 
question should be shifted to the adequate experimental design of a well-inte- 
grated study. An apparent exception to this generalization is the case where an 
issue has become so well crystallized through discussion and controversy that 
almost everyone is ready to take sides for or against, as in a presidential election. 
Even here the single question has its limitations. 


Many errors in the evaluations of findings can be seen in the following 
specific failures to provide a relevant context for interpretation. 


The failure to ask related and dependent questions. 


In any problem it is essential to ask the related and dependent questions. 
Unless we know the many qualifications and the range of attitudes, we easily fall 
into the error of misinterpreting the attitudes of approval or disapproval. People 
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themselves frequently respond with ‘‘It depends” when they are confronted with 
a yes-no type of question. To reflect their attitudes correctly it is necessary to 
inquire into the dependent circumstances which condition opinion. 


Survey findings taken before Pearl Harbor have been interpreted to mean 
that the American people were precipitated into the war by their government 
against their will. But Cantril’s trend studies clearly show that long before Pearl 
Harbor the majority of the nation favored warlike measures on our part as long 
as they stopped short of a declaration of war. People were willing to participate 
in the war on this basis but their attitude against formal and full participation 
depended upon no overt attack being made against us. Similarly a number of 
surveys report that the great majority of the American people are in favor of 
compulsory military training during peacetime. In many cases, however, this 
may mean that the individual favors such training if it does not interfere with 
the vocational or professional education of the trainee, if his own son is not to be 
taken out of his home at age 17 or 18, if it does not increase taxes, if universal 
conscription rather than the training of a few technicians is the sounder military 
preparedness, if the U.N. proves inadequate. 


The National Opinion Research Center has demonstrated this point neatly 
in its release of June 24, 1945. This report reads as follows: 


“What the public thinks about peacetime conscription depends in large measure 
upon the way the problem is presented. A survey just completed by the National 
Opinion Research Center, University of Denver, shows that 56% of the people of 
the United States think an annual federal appropriation of two billion dollars would 
be better spent on regular education than on military training. ... 

ae RC’s interviewing staff talked with a national cross-section of civilian 
adults in every section of the United States—men and women, young and old, rich 
and poor, city residents, townspeople, and farmers. All were asked: 

“Would you be in favor of or would you be against a law that would require 
boys to take a year’s military training after the war when they become 18 years old?” 








Favor military training .............. 72% 
Oppose military training ............ 20% 
Undecided 8% 

100% 


“If the government had two billion dollars a year to spend, would you rather 
have it spent on a program of better regular education in the schools, or on a pro- 
gram of compulsory military training in the Army or Navy?” 








Regular education ...........s:s00 56% 
Compulsory military training... 25% 
Neither or both .........ccccceseseeee 11% 
Undecided 8% 

100% 


The range of opinion needs to be represented adequately for proper evalua- 
tion of results. The range or degree can be found by providing many alternatives 
or by asking additional questions. Failure to explore the whole continuum can 
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distort opinion findings. For example, in October 1945 the Psychological Cor- 
poration asked this question of a cross-section of the urban and small town 
population: 


“If a man was paid $50 a week for 48 hours work in wartime and he is now 
working only 40 hours a week, should he still be paid $50?” 


On this basis people voted against increasing the hourly rate and the inference 
was that people favored-no raise in wage rates. But the American Institute with 
more detailed questioning came up with a more adequate answer. Its first ques- 
tion (incidentally stated more objectively than the above phrasing) was: 


“Because of loss of overtime, the total weekly pay of many factory workers is 
less than it was during the war. So that their total weekly pay will be the same as 
it was during the war, these workers want a 30 per cent increase in their hourly 
rates. Do you think they should or should not receive this increase?” 


Then Gallup went on to ask those opposed to a 30 per cent increase whether or 
not they would favor a 15 per cent increase of hourly rates. The results showed 
a clear-cut majority in favor of an increase of either 15 or 30 per cent. In other 
words by getting a fuller range of answers with an additional question, the mean- 
ing of the findings changed. 


The range of attitudes can also be explored by the use of an attitude scale 
which may present four or more alternative positions. Because of the difficulty 
of people remembering the many alternatives in the attitude scale, the device is 
commonly used of printing the alternatives on a card which can be handed to 
the respondent. Roper has employed this technique more extensively than other 
poll operators. A four-step attitude scale does not necessarily insure the adequate 
representation of the range of opinion, but it is an attempt to get away from the 
over-simplification of issues of the single polling question. 


The failure to determine the respondent’s own frame of reference. 


Asking limited and dependent questions makes possible the discovery of 
the respondent's attitude, but we still may not know enough of his own reasons 
to be sure of the frame of reference in which he gave his answers. People answer- 
ing “Yes” to the same question do not necessarily belong in the same category. 
Their reasons may vary decidedly. For example, a national network recently asked 
a cross-section of the American people whether or not there should be any gov- 
ernment control over news broadcasts over the air. A majority favored control 
but when they were asked for their reasons it became clear that there was a funda- 
mental difference in basic points of view. One group wanted government control 
in the interests of military and national security. Another group wanted control 
not in the interests of censorship but to make sure that private enterprise should 
not control the sources of information. The two groups, both answering “Yes” 
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to the question, were thinking in such opposed frames of reference that their 
answers could have been easily misinterpreted if their reasons had not been ascer- 
tained. A “reason why” question may often be too direct to get at the frame of 
reference of the respondent but some procedure, either direct or indirect, is neces- 
sary for interpreting the meaning of people’s responses. 


In general, the open-ended type of question is the basic tool for the deter- 
mination of the individual’s frame of reference. This permits the respondent to 
answer in terms of his own attitudinal orientation. The dimensions of the prob- 
lem that seem important to him can thus be discovered. In a large-scale survey it 
is not necessarily desirable to make all questions open-ended but it is generally 
wise to retain a few open-ended questions and it is, of course, essential to use such 
questions extensively in the pretest stage of the questionnaire. 


The error of imputing absolute value to a single percentage. 


The use of single questions in opinion surveys leads to over-generalization 
from the percentage of yes or no answers. People are categorized on the basis of 
their answers to a question and the number of them assigned to a given category 
determined by the percentage approving one specific question. For example, in 
the February issue of Fortune (1946) the conclusion is reached that 8.8 per cent 
of the American people are anti-Semites. 


To quote: 


“Identified as anti-Semites are those respondents to a survey questionnaire who 
took one or both of two opportunities offered to express hostility to Jews. The 
questions that elicited the latent hostility of the 8.8 per cent were: 


“Are there any organizations or groups of people in this country who you feel 
might be harmful to the future of the country unless they are curbed?” (If “Yes’’) 
Name: (Here 5.1 per cent named Jews) 


“Are there any groups of people you think are trying to get ahead at the expense 
of people like you?” (If ‘Yes’”) Name: (Here 6.5 per cent named Jews) 


“Once identified, the anti-Semites can be examined by means of their replies to 
other questions in the same survey.” 


To infer the absolute amount of anti-Semitism from two questions is a 
debatable procedure. Merely because 91.2 per cent of the people did not name 
the Jews on either of the two questions does not establish their complete lack of 
anti-Semitism. Anti-Semitism is a complex set of attitudes and practices which 
varies both in intensity and in qualitative aspects. The categorizing of 8 per cent 
as anti-Semites implies that everyone is either an anti-Semite or not one at all. 
Obviously, if other types of questions had been used the percentages might have 
been higher or lower, depending upon the questions employed. To say that there 
is 8.8 per cent of anti-Semites in the United States is to assume that we have a 
single yardstick to measure this complex attitude and that we know its zero point. 
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Relative rather than absolute measures are scientifically more justifiable. For 
example, trend studies in which questions are repeated over a period of time 
furnish a relative notion of the increase or decrease of sentiment or prejudice. In 
all fairness to the Fortune Survey, it should be stated that it did report relative 
findings in the trends in anti-Semitism over a period of time. 


Another way of presenting relative findings is to compare various groups 
sampled within the same survey. A good example of this can be seen in the 
study of German morale made by the U.S. Strategic Bombing Survey of the War 
Department. The objective was to determine the effect of bombing upon German 
civilian morale. Since there is no absolute yardstick of morale with a fixed zero 
point, the study was designed to compare German cities subject to various de- 
grees of bombing on the many dimensions of morale. If relatively more people 
showed high morale in unbombed than in bombed places, then the finding was 
meaningful in spite of the lack of an absolute scale. As a matter of fact, in this 
study the highest morale was found in unbombed communities but there was just 
as much loss of morale in cities suffering medium bomb tonnage as in heavily 
bombed cities. 


The Misleading Use of "Typical" Cases and "Typical" Comments 


Many of the errors described so far are inherent in the usual commercial 
polling approach to research, but the more scientific work of government agencies 
and foundations is not without its weaknesses. To enable the administrator to 
visualize the findings, research results have often quoted “typical” cases and 
“typical” comments from respondents. These comments and cases make the 
people whose reactions are being reported living personalities and are a good 
device for communicating feeling, but they belong more to the literary than the 
scientific approach. The objection is not so much to the use of case material but 
to its presentation as typical when no rigorous procedures have been employed to 
establish the typicality of the quoted material. In fact, the tendency is to select 
excerpts from interviews which are unusually cogent, colorful and dramatic. 


For example, the Surveys Division of OWI and the Division of Program 
Surveys of the Department of Agriculture both used case materials in presenting 
their findings to give administrators some feel for the human aspects of the 
problem. In one study of worker morale in the shipyards, emphasis was placed 
upon the necessity of a feeling of participation by the shipyard worker himself. 
One of the older workmen was quoted as follows: 

“They used to let more men go on the trial run of the boats. Now they have 


cut that out and a few of the big shots go and have +, and get drunk. All the 
men want to get to go on these trips, but they stopped it. The men feel that now. 
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They are usually gone about twelve hours. It would help create a better feeling 
among the men if they would let them go. We launched a boat Sunday. My foreman, 
who has worked there 27 years, was there at the launching and he and his wife and 
baby stood out in the rain to watch it. Those big shots who stood up there under 
the shelter don’t know anything about building a ship. I think those who do the 
work = get the credit for it and not those big shots who make all those pretty 
speeches.” 


Again, in another study, when rent controls were first put into effect during 
the war, the reaction of those approving it was expressed in the words of a young 
coppersmith in the Philadelphia navy yard. 

“I think it is one of the best moves they ever made here. The rents were going 
up something terrible. They were taking advantage of the times. They even did it 


in our own case—trying to soak the guy whose wages had gone up a little. If we 
didn’t have rent control, eventually we’d have to sleep in the park.” 


It can be argued that these utterances selected for presentation in the final 
report, though voiced by the articulate few, really represent the feeling of the 
inarticulate many. But this argument to be tenable must be documented by 
specific supporting evidence. 


The general usage is not to claim explicitly that case materials and comments 
are typical of all respondents. Quotations are generally presented as expressions 
of particular people. None the less, the reader will often infer that the dramatic 
comments presented do typify a whole class of respondents. 


The Weaknesses of the Quota-Control Sample of Most Polls 


The excellent record of the polls in predicting election returns has been 
taken as evidence of the representativeness of their sample for other purposes. 
Thus when we are told the percentages of the nation disapproving a loan to 
Britain, and when this national figure is broken down into the percentage of 
farmers, workers, business men, middle-Westerners, New Englanders, and other 
sub-groups, we accept the accuracy of the figures because the polls were within 
2 or 3 per cent or less of predicting the last presidential election. 


But the accuracy of the national sample is no guarantee of the accuracy of 
the sub-groups within the sample because the usual quota-control method of the 
polls is not designed to control the representativeness of the sub-groups. The 
general rule in the quota-control method is to check the percentages of types of 
respondents in an over-all way, but not the distribution of these proportions 
within sub-groups. Thus the interviewers in a given community are told to bring 
back the correct proportion of low-income people, the correct proportion of old 
people, etc., but they are not told how many of their poor respondents should be 
in a given age group. This does not necessarily produce distorted sub-groups but 
it permits such distortion to occur. 
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Whenever interviewer-selectivity of respondents is permitted to operate over 
a wide range as in the quota-control method, biases in the sample can result. The 
most easily identified bias is the under-representation of the poorer economic 
groups. The polling interviewer (frequently a middle-class housewife), though 
given the proper quota of poor people, in practice tends to avoid the inarticulate 
and the uninformed and to neglect the dirty streets and dingy houses on the other 
side of the tracks. An analysis of polling samples reveals between 12 and 20 per 
cent more people in the better educated groups than the U. S. census reports. Since 
this bias is now known it can be corrected by weighting up the underrepresented 
lower economic group. Generally, poll results are reported without this weight- 
ing. This means that the percentages favoring conservative measures are too high 
for the nation as a whole. Moreover, with a wide margin of interviewer-selec- 
tivity of respondents other biases may operate which are not as easily identified 
as the socio-economic seleciion. 


Over-All Evaluation 


In concentrating upon the errors of interpretation in survey findings, this 
discussion may itself be misinterpreted as a negation of the value of public opinion 
polls and social surveys. But the fact that findings are sometimes inadequately 
and inaccurately presented does not destroy the essential validity and tremendous 
value of surveys. In fact, polls and surveys may well constitute the most sig- 
nificant recent development in social science. By the extension of quantitative 
methods into areas of human relations we are beginning to build a true science 
of social behavior. Compare, for example, the old intuitive guesswork that once 
passed for political science with the exact knowledge of voting behavior and its 
determinants, now accumulating. The many criticisms and cautions in the use of 
survey methods are an indication of the growing complexity and development of 
scientific procedures in this field. 


Finally, it should be emphasized that radical changes in methodology have 
taken place since the polls first began using a single yes-no question to attack an 
important problem. The progress has been in the direction of comprehensive 
scientific surveys emphasizing intensive pre-testing, experimental study-design, 
and precision sampling. The criticisms presented here are in the interest of secur- 
ing wider recognition and use of these advances, and a more adequate use of 
survey findings. 














How Surveys Are Made 


ELEANOR E. MACCOBY AND ROBERT R. HOLT 


The surveying of public attitudes and opinions during recent years has at- 
tracted an increasingly large audience. The ‘‘public opinion polls” now share 
newspaper space with the most ominiscient columnists and during pre-election 
weeks they become front-page news. 


To most people, who know about polls only what they read in the news- 
papers, surveying must appear to be a transparently simple procedure. The ques- 
tions asked seem obvious enough and the percentages always add to a hundred. 
They seldom suspect the tortuous and detailed labors which lie behind the neat 
columns of figures. For many such followers of public opinion the final results 
are doubtless the only part of the process of surveying that has any interest, but 
there are others who are not so easily satisfied. 


__ This article is meant for those people, not themselves expert in surveying, 
whose interest in public opinion includes a curiosity as to how the surveys are 
made, and perhaps an interest in the possibilities of using the polling procedure 
in their own organizations or communities. It is not intended as a manual of 
instructions for beginning pollers; no short article could serve such a purpose 
adequately. It proposes to describe briefly the major steps that are followed in 
surveying and to answer thereby some of the common questions as to the tech- 
niques which lie behind survey findings. 


Defining the Objectives of the Survey 


The first step in any survey, as in any other planned observation, is to define 
the question which the survey is to answer. The more clearly the objectives of 
the study are specified, the more likely it is to yield clear-cut results. 


Usually the definition of objectives begins with a broad, general statement 
of the problem and then turns to a listing of all the items of information the 
survey will gather. The survey director must analyze his problem carefully to 
make sure that he includes in his study as many of the pertinent aspects of his 
problem as he can. It is his responsibility to find out as much as he can in this 
planning stage about the important factors bearing on the subject of the study. 


Suppose we consider as an example the problem of conducting a survey on 
public attitudes toward taxes. It is necessary to define the problem by specifying 
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what varieties of taxes the survey will cover. Attitudes toward federal taxes may 
be different from attitudes toward state and local taxes, and the survey may deal 
with all or only some of these. If the survey is to deal with both income taxes and 
sales taxes, a distinction must be made between them in the detailed plans for the 
survey. The administration of tax policy would be an important consideration ; 
public reactions to the tax forms, dates and methods of payment should be 
studied. An effort might be made to find out the extent to which people would 
be willing to see public services curtailed for the sake of reducing taxes. Full 
understanding of the attitudes in this area would require data on the level of 
public information about taxes. Do peopl: know what the tax rates are? Do 
they know what the money is used for? The survey should also cover the rela- 
tion, if any, between attitudes toward taxes and other attitudes and personal 
attributes of the people interviewed. Do attitudes toward taxes differ in different 
income and education groups? How are they correlated with attitudes toward 
other governmental activities ? 


The survey director must consider all these aspects of his problem. It may 
be that he will not be able to deal with them all, because of limitations of time 
or money and he will therefore have to limit the scope of the survey. His goal is to 
specify what aspects of the subject he will cover, and to anticipate exactly what 
tables he will want in his final report. 


Choosing the Study Design 


While most studies require a single survey of one particular group, this is 
not always the case. Surveys may be designed in a variety of ways, depending on 
the objectives of the study. It may be desirable for some purposes to survey two 
contrasting communities or industrial plants and to compare the two sets of 
results. In studies where it is important to measure changes or trends in opinions 
or behavior, a group of people may be selected as respondents and this group may 
be interviewed several times at specified intervals. When studying cause-and- 
effect relationships it is sometimes possible to use experimental techniques. To 
study the effects of a certain motion picture on attitudes, for example, two similar 
groups might be selected, an “experimental” group who would be shown the 
film and a “control” group who would not. The influence of the picture on the 
attitudes of the experimental group could then be studied. 


Selecting the Sample 


As part of the basic planning of the survey, the survey director must define 
exactly the group to be covered by the survey. If he is studying the opinions of 


46 











the American people on a certain issue, the group to be studied (the “universe” ) 
will probably be the entire adult population. If he is making a morale survey 
within a certain factory, the universe might be all the employees in the factory. 
Or perhaps the survey is to cover only the skilled, semi-skilled and unskilled 
workers, not the clerical and managerial staff. 


When the group to be studied has been decided upon, the next question to 
be answered is: Can the survey include a contact with each member of the group? 
In a study of employees’ morale, it may be possible to interview every employee. 
In a nation-wide public opinion survey, however, it is obviously impossible to 
poll each adult person in the country. More often than not, a survey must be 
based upon a sample of the universe. Some individuals must be selected from 
the universe in such a way that they will represent all the people in the universe. 


The most reliable way to choose the individuals to be included in a sample 
survey is to use some random method of selection. Selecting at random means 
using some automatic method of choosing which gives each individual in the 
group to be studied an equal chance (or at least a known chance) of being in the 
sample. To take a random sample of the members of an organization, for ex- 
ample, one might take every tenth card in the membership file. To decide which 
card to take as the first case, one might open a book at random and take the last 
digit of the page number. Experience in survey work has demonstrated that if, 
instead of selecting at random, a surveyer tries to pick a representative sample by 
choosing certain people or places that he believes are representative, a biased 
sample will probably result. When surveys are based on random sampling, the 
probable size of the sampling error can be computed mathematically. The errors 
in a sample which has not been selected by random methods cannot be estimated 
in any precise way. 


There are many pitfalls in choosing a random sample, many ways in which 
bias may creep in and prevent the selection from being truly random. Suppose 
interviewers have been sent to certain blocks in a city and told to visit every fifth 
dwelling on each block.. In counting dwellings, there is danger that the inter- 
viewers will miss alley dwellings, basement apartments, servants’ quarters over 
garages, and other inconspicuous households. It is evident that if this happened 
the sample would be biased in the direction of having too few people in the 
lower income groups. 


Bias may be created when no attempt is made to include in the sample those 
people who are hard to find at home. If the interviewers take all their interviews 
in households where they find someone home the first time they call, it is clear 
that people who are not home much do not have an equal chance of coming into 
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the sample. It is well known to surveyers that people who stay home a good deal 
differ as a group from people who do not. 


When samples are taken from lists of names there is danger that the list will 
be incomplete. This was apparently partly responsible for the miscarriage of the 
Literary Digest poll of the 1936 presidential election. The sample used by the 
Digest, purporting to represent all voters, was taken from such sources as lists of 
telephone subscribers. This meant that people without telephones (on the aver- 
age from a lower socio-economic level than people with telephones) were not 
represented adequately. An additional bias was probably introduced by the fact 
that the poll depended on people mailing in the post-cards which they received. 
The people who mailed in their cards were probably not comparable to the people 
who did not. 


Bias may also be introduced into a sample by allowing the interviewers free- 
dom in the choice of the people to be interviewed. In some polling operations, 
interviewers are told the number of interviews to take, and they are told that 
these interviews must be distributed in certain ways. For example, an interviewer 
might be instructed to take one-half of his interviews with men, the other half 
with women; one-tenth with Negroes, nine-tenths with whites; and one-fourth 
of the interviews from each of four income groups. Aside from these restrictions, 
the interviewer has freedom of choice, and it is evident that he could follow 
instructions and still interview only ‘‘available’’ people—waitresses, barbers, 
policemen, people at railway stations, people who sit on their front porches or 
stroll in the park, and so on. It is clear that with this procedure, certain groups 
in the population may be under-represented. 


Whenever sampling is done by a system of random selection, the more cases 
the sample has, the more likely it is to represent the universe well. The reasoning 
behind this is as follows: Suppose a survey is being done of opinions about 
foreign affairs. This is a subject on which people with different amounts of 
education differ markedly, so it is important to have college graduates, high 
school graduates, grade school graduates, people with some grade school, and 
people with no formal education at all. If only three cases were chosen, they 
obviously could not adequately represent these five educational groups. A sample 
of forty cases might easily, by chance, contain four college graduates, or it might 
contain none—there is not a very good chance that the true proportion of college 
graduates (perhaps five percent) would be obtained. The larger the number of 
cases, however, the better the chances that all the levels of education will be 
represented in their proper proportions. It might be argued that one should 
deliberately select the right number of people from each educational level and 
consider this as a representative group. But it would be representative only with 
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respect to education, and there are many other characteristics which are related to 
opinions on foreign affairs, some of which would not be known in advance. A 
large sample randomly selected assures a sample which will be reasonably repre- 
sentative of all characteristics of the people in the universe. 


When random methods of selection are followed, increasing the number of 
cases will improve the sample by reducing the sampling error, but mere increase — 
in the number of cases will not correct for a bias in the sample. When a sample 
is “biased’”’ its errors are not chance errors which tend to cancel each other out 
but are systematic and create deviations in the same direction. For example, 
in a survey of income and savings the results would be greatly affected by the 
inclusion or exclusion of a few millionaires. By chance a sample might contain 
too many millionaires or too few. If this error were purely a matter of random 
sampling error, an increase in the number of cases would provide a better chance 
of getting just the right proportion of millionaires. But suppose millionaires 
could not be interviewed because the interviewers could not get past the butlers 
and secretaries, or because the millionaires were out hunting in the Maine woods 
or tarpon fishing off Florida; then all the errors would operate in the direction of 
including too few millionaires rather than too many and the sample would be 
biased. Increasing the number of cases in the sample would not reduce the error, 
since the same cause of error would affect the new cases and in the same direction. 


In deciding how many people will be included in his sample, the survey 
director is usually influenced by considerations of economy. His purpose is not 
to use the largest possible sample but rather to use the smallest sample which 
will give results of acceptable accuracy. Sampling experts have worked out 
formulae for estimating the sampling error which is involved in samples of 
different sizes. The survey director chooses, then, a sample size which will have 
a sampling error small enough for his purposes. 


Among the factors which determine the number of ases needed for a 
sample survey, the following are perhaps the most important: 


1. The desired accuracy of the survey results. If the survey director wishes to be 
reasonably sure that his final figures are accurate within one percent, he must 
have a larger sample than if he is willing to accept a margin of error of five 
percent. 

2. The variability of the characteristic to be measured. It would take more cases to 
sample for a variable like income, which has a wide range and many different 
values, than to sample for variables like age or sex. 

3. The desired breakdowns of the findings. Fewer cases are needed in a national 
survey, for example, if the findings are to be used only as national estimates than 
if they are to be broken down by state or region. In the latter case, it would be 
necessary to have enough cases in each state or region to represent it separately, 
while for national figures alone this would not be necessary. 


When the objectives of a survey call for a national sample, it is seldom 
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possible to select respondents by taking, say, every ten thousandth person in the 
country. Travel expenses for the interviewers would be too great; some method 
must be adopted to reduce the number of communities in which interviews are 
taken. Usually this is done by first choosing a sample of counties and then select- 
ing a sample of people to be interviewed within these counties. In selecting the 
counties where interviews will be taken, the survey director can cut down the 
sampling error by “stratifying” the sample. This means simply that he will 
arrange all the counties of the country in order according to some characteristic, 
(such as percent of Negro population), divide these ordered counties into strata 
(high, medium and low), and select sample counties at random within each 
stratum, thus making sure that a proper proportion of counties with high, medium 
and low Negro populations will come into the sample. 


The “modes of stratification” which aie used are always characteristics 
which are thought to be related to the subject matter of the survey. It would be 
possible, for example, to stratify counties according to their average annual rain- 
fall, so that a proper proportion of wet and dry counties would be included in 
the sample. But, to continue a previous example, if the survey were measuring 
attitudes toward our foreign policy, the sample would not be improved by this 
stratification since such attitudes are presumably not related to rainfall. A strati- 
fication of the counties according to the average educational level might, however, 
improve the sample, for if education tends to be related to attitudes toward world 
affairs then by making sure that the sample contains a proper proportion of 
counties that are high, medium and low in educational status the chances are 
increased of obtaining a proper representation of people of different points of 
view toward foreign affairs. 


Writing the Questionnaire 


With these phases of study planning completed, the survey director proceeds 
to write the questionnaire itself. First he must decide whether he will use any 
“open” questions, or whether they will all be of the “closed” or “polling” variety. 
Polling questions, the kind most frequently used in the ordinary Gallup and 
Fortune polls, are questions such as the following: 


“If the election were held today, who would you vote for—Roosevelt or Dewey ? 
Roosevelt .............+ Dewey .......cccseeeee Don't know ............000 


In questions of this sort, the interviewer simply checks the choice which his 
respondent makes. Polling questions can be more complex than this, of course: 


“Which of these comes closest to expressing what you would like to have the 
USS. do after the war?” (July, 1945, Fortune) 
. Enter into no alliances, and have as little as possible to do with other countries. 
nd only on separate alliances with certain countries. 
4 Take 7 =_ part in an international organization. 
4. Don’t kn 
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In using questions like the one above, the interviewer may either read off the 


alternatives, or he may hand the respondent the list of possible answers and ask 
him to choose one. 


In contrast to these kinds of questions, ‘‘open’’ questions do not present the 
respondent with fixed alternatives from which he must choose, but ask a general 
question which the respondent may answer in his own words. The following 
are examples of open questions: ‘What do you think will happen to prices of the 
things you buy in the next year or so?” “Why do you think so?” ‘‘As you see it, 
what is the main thing the U.N. is set up to do?” When questions of this sort 
are asked, the interviewer must write down each respondent’s answer as nearly 
verbatim as possible, and these answers must be grouped into categories later. 
When polling questions are used, the interviewer's job is much easier, and the 
answers are easier to handle in analysis later on. 


Polling questions are usually used in surveys to obtain enumerative mate- 
rial, such as age, education, nationality, and the like, or simple statements of fact 
(Do you own a radio? Are you a registered voter?). They are also frequently 
used for surveying attitudes if the survey deals with well-understood issues on 
which people have clear-cut opinions. But in an area where attitudes are complex 
and confused, there is danger that polling questions will yield misleading results. 
When open questions are used, people express their views together with any 
reservations or contingencies which are present in their minds; when they are 
presented with a polling question and asked to choose one of the alternatives, 
they may not have an opportunity to express their reservations, unless specific 
additional questions are asked to bring them out. Polling questions have certain 
disadvantages, too, when it comes to asking people to make suggestions for im- 
proving a situation which they dislike, or to give reasons for their beliefs. If 
check-lists of alternatives are presented to them, there is the danger of suggesting 
ideas which were not actually present in their minds. There is also the danger 
that not all the great variety of possible reasons and suggestions will be included 
among the poll’s alternatives. A common practice in questionnaire construction 
is to combine polling and open questions. For example, polling questions to 
which the respondent must answer “Yes” or “No” are often followed by the 
question “Why do you think so?” and the interviewer then writes down the 
respondent’s reason in the blank provided. 


In writing his questionnaire, the survey director must give careful attention 
to the order in which different topics and particular questions are taken up. The 
opening questions must be such as to interest the respondent and stimulate his 
cooperation. The sequence of questions must be orderly, and logical transitions 
must be made from one topic to another. The context in which a question is 
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asked can have the greatest influence on the answers to it. Likewise the order and 
number of alternatives can affect the results greatly in a question where the re- 
spondent chooses one of a set of alternatives as his answer. To take care of this 
.. difficulty, interviewers are sometimes instructed to vary the order in which they 
present alternatives, or sometimes different forms of the questionnaire are pre- 
pared, giving the questions in a different order. 


The so-called “funnel” arrangement of questions is often useful. In this 
procedure, a very general question is followed by one in the same area which is 
somewhat narrower, and this in turn is followed by a more specific question. This 
technique permits the respondent to answer the general question spontaneously 
before any specific aspects of the problem have been suggested to him, but 
nevertheless pins him down later on specific points. Suppose, for example, that 
a study is being done on consumers’ cooperation with the Government’s food 
conservation program, and the study calls for a table showing how many people 
are using less bread than they normally would. People may be asked first: “Are 
you personally doing anything to conserve food?” and if they say they are doing 
something, they may then be asked “What are you doing?” The answers to these 
questions will give evidence on what aspects of the food conservation program 
are upper-most in people’s minds, but some people may fail to mention conserva- 
tion of bread even though they are actually eating less of it. To get specific 
information, the survey could proceed to a direct question on whether the re- 
spondent was conserving bread, and if necessary, could then include questions on 
the different ways of saving bread. 


The task of writing the questions themselves is a difficult one. The writer's 
first objective must be to make sure that his question is understood. Its wording 
must be clear and unambiguous, and the words used must be simple enough so 
that they will be understood by the least educated of the respondents. There are 
certain regional variations in the use of words which the writer must keep in 
mind if his question is not to mean different things in different parts of the 
country. In addition the writer must avoid referring to particular ideas, policies, 
recent events, or personalities, unless he has some assurance that the respondents 
will be familiar with them. It is hardly necessary to point out that, in order to be 
understood, a question must be reasonably brief. If a question contains two or 
three long sentences, the respondent will often forget what the first part of the 
question was before he gives his answer, so that he actually responds only to the 
last few words of the question. 


Each question should have a single focus. If it contains several ideas, it is 
impossible to tell what part of the question the respondent's answer refers to. 
For example, the question ‘‘Do you think a man would be wise to put his money 
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into real estate and securities these days?’ is poorly worded, for one man might 
answer ‘Yes’ when he believes that real estate is a good investment and securities 
a poor investment, while another man’s “Yes” might signify approval of securities 
but not of real estate. 


The writer of questions must keep in mind the fact that the use of prestige 
words or other emotionally-toned words may materially affect the responses to a 
question. It is well known, for example, that attaching a prestige name such as 
Roosevelt’s to a policy proposed in a question will increase the proportion of 
respondents who express approval of the policy. Similarly, it is almost certain 
that more people would say “Yes” to: “Do you think the United States should 
send food to the starving people in Europe?” than would agree if the word 
“starving” were omitted. 


The survey director must be careful in the use of emotionally-toned words, 
but there are occasions in which he may find it desirable to employ them. If he is 
studying opinions about sending food to Europe, he may deliberately use the 
word “starving” because many people in Europe are, in fact, starving, and because 
the publicity on the food crisis appeals to people on the grounds that they must 
help starving people. To omit the word, then, might be to underestimate the 
number who would be willing to share American food under the conditions 
actually prevailing. Furthermore, the survey director sometimes finds it desirable 
to include an emotionally-toned question with the specific purpose of finding out 
how many people hold their opinions so firmly that they cannot be swayed by 
devices of this kind. 


The question of the effect of emotional “loading” on the answers to a ques- 
tion leads to the more general problem of bias in questions. The point is often 
raised: Do the answers obtained in a survey represent the “true” attitudes which 
people have, or have the attitudes been distorted by a “leading” question? In 
survey research, every attitude must be studied through the answers to questions, 
and no answer is free from the influence carried by the wording of the question 
which was used. Every question is “leading”, in the sense that it at least specifies 
the subject-matter about which the respondent is being asked to talk. 


For these reasons it is important that each survey finding be interpreted in 
the light of the particular question which was used. Findings must not be loosely 
generalized to cover whole areas of opinion. Particulatly when a study deals with 
attitudes which are complex, it is important to ask a battery of questions bearing 
upon different aspects of the problem, so as to achieve what an earlier article in 
this issue calls a “well defined context for interpretation.” 





1Cantril, H., Gauging Public Opinion, Princeton, Princeton Univ. Press, 1944, Chapter 
Il. 
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Although the problems of question-wording need to be emphasized, the 
survey director in writing his questionnaire can take comfort from the fact that 
in many cases small differences in question-wording produce relatively little 
change in public response. Especially when the questions are about facts which 
are well known to the respondents, it is surprisingly difficult to affect the answers 
much by context, question order, or the wording of the questions. 


Pretesting the Questionnaire 


It is important to give every question a test run before using it. Often ques- 
tions which appear satisfactory when they are written turn out to be too difficult 
or ambiguous, or they unexpectedly set off irrelevant trains of thought on the part 
of the respondents. Questions can be tested for defects of this sort by trying 
them out on a representative group of people. The answers given on the pretest 
are taken down in full, and the respondents are asked to explain what they mean 
by their answers. This procedure enables the survey director to detect questions 
which are being misunderstood. Often several forms of a question must be tried 
out before the best wording is found. In the pretest, variations in question order 
may also be tried out, until a smooth sequence is achieved. 


The pretest permits the survey director to check the answers he is getting 
against the objectives of the survey, to see whether the kind of information being 
assembled will solve the problems that underlie the survey. Questions must be 
discarded if it is found that they merely sound interesting but do not contribute 
anything to the objectives. 


Administering the Survey 


The survey director must make sure that his interviewers are well trained before 
they begin work, and arrange for their supervision throughout the interviewing 
process. Poor interviewing can ruin a survey even though the planning and ques- 
tionnaire construction have been well done. 


The calibre of interviewers needed to do the job depends, of course, on the 
complexity of the subject, whether polling or open questions are used, and on 
the amount of freedom which the interviewers will be allowed in the interview- 
ing situation. A certain amount of judgment on the part of the interviewer is 
always required. Respondents frequently make replies which are not answers to 
the questions at all. In these cases the interviewer must recognize this and repeat 
the question. Interviewing can never be completely mechanical, if for no other 
reason than that the interviewer must learn ways to gain the cooperation of the 
respondent before he can begin the formal interview. For intensive interviewing, 
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in which the interviewer is allowed to adapt his questions to the individual case 
to some extent, a high degree of skill and training is required. For most surveys 
interviewers must be personable, intelligent, and tactful. 


Perhaps the most important basic principle an interviewing staff must be 
taught is not to influence the answers of the people being interviewed. This 
means that they must learn to avoid expressing disapproval or approval of any- 
thing the respondent says. When they have some latitude in the rewording of 
questions they must ask them in a non-directive way. For example, it should 
become habitual to ask “Are you working now?” instead of “You're not working 
now, are you?” Similarly, in using open questions in which a respondent gives 
reasons for his opinion, the interviewer must learn how to stimulate discussion 
without slanting it. He may try to get the person to express himself more fully 
by the use of such questions as “Why do you think so?” and ‘Just how do you 
mean that?” but must avoid suggestive questions such as: “Is that because of 
the high cost of living?” 


The quality of a survey can always be improved by devoting a good deal of 
attention to training the interviewers on the specific subject-matter of the survey. 
There are almost always certain terms which need to be very clearly defined in 
the interviewers’ minds. For example, if an interviewer must check whether or 
not the respondent is employed, he must learn to know how to classify people 
who are employed part-time. If he is to include only farmers in the survey, he 
must have the term “farmer” defined so that he will know whether or not to talk 
to nurserymen, seasonal farm laborers, small farmers who work part time in the 
city, and so on. 


Analyzing the Results 


When the interview schedules have been filled out and sent in to a central 
office, the survey director must tabulate the answers in some way so that the survey 
results will be summarized and easily understood. The simplest way would be 
to go through the schedules and tally the answers to each question, so that a count 
would be obtained. This system is not very convenient for comparisons of groups 
within the sample, however. To find out how men compare with women on a 
certain question, it would be necessary to divide the schedules into two groups 
for men and women, and tally separately. If a count by income groups were 
desired, a new grouping of the schedules and a new tally would be required. 


For large-scale operations where internal comparisons will be needed, it is 
usually found to be most convenient in the long run to record the answers on 
punch-cards. Different answers are numbered (or ‘‘coded”’) ; each respondent has 
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a separate card, and on this card are punched the numbers which represent his 
answers to all the questions. After the punching has been done, the process of 
counting the different answers is greatly simplified, for the sorting and counting 
machines will sort the cards into any desired groups and count the answers auto- 
matically. 


Coding is a fairly simple job when the respondent has been presented with 
a group of alternatives from which he must choose; each of the alternatives can 
be givert a number, and the cards may be punched immediately from the ques- 
tionnaires which have been filled out. When full narrative answers are given to 
open questions, however, coding is more difficult. Suppose, for example, that 
people have been asked to give their reasons for their opinions on a certain issue. 
A great variety of reasons will be given, and the reasons will be worded in many 
different ways. These reasons must be grouped into a limited number of cate- 
gories, and each category must be numbered for purposes of punching on the 
cards. 


It is the responsibility of the survey director to present his findings in such a 
way as to prevent unsophisticated readers from coming to unjustified conclusions. 
The problem of misinterpretation of findings very often arises when two per- 
centages are being compared, as in the table below:1 


Attitude toward continuation 














of price ceilings: March, 1945 March, 1946 
Favorable 68% 74% 
Unfavorable 25% 23% 
NI MANNED av sxscsnpvccsvavbipnsnetavcvesve 7% 3% 

100% 100% 


The question is, has there been a real change in attitudes toward price ceil- 
ings? Assume for the moment that the survey reporting these findings had a 
small sample with a sampling error of five percentage points. Under these cir- 
cumstances, it is quite possible that the true percentage favoring continuation of 
price ceilings during the two years is, say, 72 percent, that there has been no 
change in this percentage, but that the two figures which were obtained differed 
from 72 percent because of random sampling error. A large increase would be 
needed before one could feel confident that a real change in sentiment has 
occurred. On the other hand, if the survey were based on a large number of cases, 
and had a sampling error of only one percentage point, the difference between 
the percentages shown above could be relied on as showing a real difference, not 
just a chance one. There are statistical formulae by which the survey director can 





1This table does not represent actual survey findings, but is simply included here for 
illustrative purposes. 
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compute the probable range of error of his percentages, and he must test his 
differences for reliability before presenting them in his report. 


In presenting his findings, it is also the responsibility of the director to cau- 
tion his readers against generalizing the findings to a different population from 
the one measured by the survey. If the survey is based upon a sample of mid- 
western farmers, this fact should be emphasized, so that the reader will not assume 
that the results apply to all the farmers of the nation. 


As has been pointed out earlier, each answer must be interpreted in the light 
of the particular question asked. Answers to one question cannot be taken to rep- 
resent attitudes toward other aspects of a broad field of attitudes. For example, 
if most people say Britain ought to pay for the food which we send to her, it 
should not be concluded that the majority would recommend withholding food 
if Britain cannot pay. The survey director must not only avoid drawing unwar- 
ranted conclusions himself, but he must caution his readers against doing so. He 
cannot, of course, prevent misuse of the findings by unscrupulous readers, but he 
can minimize their misuse by well-intentioned people who will avoid pitfalls if 
they are only pointed out. 


Conclusion 


The preceding pages outline the steps which are followed in surveying public 
opinion. The reader may conclude from this description that making a survey is a 
very complicated business indeed. So it is, and even more so than indicated if one 
is dealing with a large-scale study of a complex issue on which opinion is con- 
fused and contradictory. However, the many precautions and special techniques 
derive from very simple principles; they are the rules of scientific method which 
underlie all careful attempts to gather facts reliably and interpret them validly. 
In its essence, a sample survey is merely a formalized procedure for making 
observations. It is a device for recording items of information in a systematic 
way, while eliminating the most common errors of bias, insufficient evidence, 
and the disregard of negative cases. 


For the non-professional student of public opinion, it is not important to 
learn the detailed methodology which has developed in the surveying field. It is 
important, however, for him to understand the basic principles of surveying, so 
that he may evaluate critically the survey findings which he encounters, and seek 
sound advice if he should desire to utilize the method. 
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A Summing-up 


ANGUS CAMPBELL 

The theory and practice of surveying have come a long way since 1936 
when Dr. Gallup made the nation’s headlines by predicting that President Roose- 
velt would be re-elected while the Literary Digest was assuring its readers that 
Governor Landon would win in a landslide. No one paid much attention at that 
time to the fact that Gallup’s prediction of the Roosevelt vote was ten percent 
too low; it was enough that he picked the winner. Since that time surveys have 
made great advances, both in the precision of their findings and in the complexity 
of the problems to which survey procedures are applied. Improved methods of 
sampling save made possible the reduction of bias and sampling error in survey 
findings ; improvements in techniques of study planning, interview construction, 
and interviewing now permit the successful analysis of problems much more 
complicated and important than the pre-election polling of the presidential vote. 


Two Reservations 


The fact that present day surveying has developed in large part from the 
early “straw vote” attempts to predict election returns has had two rather unfor- 
tunate consequences. The carry-over of “polling” techniques to the investigation 
of public response to social and political issues has tended, first, to represent 
public attitudes on these complex problems in highly over-simplified terms, and 
second, to impute an equality to the opinions of all respondents regardless of 
palpable differences in the degree of conviction with which these opinions are 
held, the extent of understanding on which they are based, and the relative polit- 
ical or economic importance of the people who hold them. Both of these facts 
have been touched upon in earlier articles in this issue but they deserve further 
comment. 


The success of the pre-election polls undoubtedly results in large part from 
the fact that voting in presidential or congressional elections is for most people 
a very simple and stereotyped decision. In all but a tiny minority of cases, the 
choice has only two possibilities both of which are closely associated with party 
labels. For a sizeable proportion of the voters the names of the candidates are 
only incidental; they know how they will vote even before the nominations are 
held. Those voters who do not follow a consistent party line do not lack for 


1Lazarsfeld, P. F., Berelson, B., and Gaudet, H., The People’s Choice. Duell, Sloan 
and Pearce, New York, 1944, v—178. 
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information about the respective candidates. The newspapers and radio give 
them ample rationalization for whichever choice they may come to. 


It is also characteristic of voting behavior that the voter cannot express any 
nuances of approval in marking his ballot; he is either for the candidate or he is 
against him. Reservations and qualifications about the candidate are of no im- 
portance. In the eventual tabulation, all of the votes count the same whether cast 
in whole-hearted approval or with reluctance and misgivings. These attributes of 
voting behavior greatly facilitate the attempt of the poller to obtain a valid indi- 
cation of the way people will vote. He has only to ask his respondents whether 
they plan to vote for this man or that and to total the number who check one 
name or the other. 


In extending their investigations from surveys of voting preference into 
larger areas of attitudes and opinions, the pollers have tended to apply the same 
techniques they use so successfully in forecasting the vote. Thus, they quite 
commonly go to the public with questions about such large issues as the proper 
size of the Army and Navy, the safest method of controlling the atomic bomb, 
or the propriety of a program of federal aid to education, and report the per- 
centages who answer “Yes” or “No” to these questions in very much the same 
way they report their straw votes on elections. This procedure creates an impres- 
sion of public decisiveness on current issues which must often be quite illusory. 
The fact that a certain proportion of the public answer “Yes” when asked if they 
think the government should turn over control of the atomic bomb to the United 
Nations cannot be interpreted as simply as the fact that a certain proportion of 
the people say they plan to vote for Wintergreen for president. Deciding how to 
control the atomic bomb is a vastly more difficult thing than deciding how to vote. 
Relatively few people have given the question much thought; even fewer under- 
stand the problems involved. Those who do hold any opinion on the matter are 
likely to have it qualified with reservations of one kind or another. Yet, the 
polling question will give the impression that the public is divided into groups 
holding categorical opinions on this highly complicated problem. 


Some writers have been moved to interpret polling findings of this type as 
expressing a “mandate from the people.” In the absence of any data as to how 
well the public understands the issues involved, how strongly they hold the opin- 
ions they express, or what qualifications they attach to their answers, this would 
appear to be a rather extravagant use of the term “mandate”. Furthermore, it is 
probably incorrect to interpret any poll of public response to a proposal for poli- 
tical action as more than an expression of the distribution of approval and dis- 
approval regarding the issue. Government by plebiscite was not intended by the 
authors of the American Constitution, and there are few who would favor placing 
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the legislative powers of government directly into the hands of the people. The 
public mandate is expressed through the election of representatives. Survey find- 
ings on political issues have an advisory rather than a mandatory function and as 
such they undoubtedly have great value in facilitating the democratic process. 


Survey findings are customarily presented in percentage figures with each 
respondent weighted equally with every other. If a survey is to serve as a refer- 
endum, it is of course consistent with our political philosophy that every voter 
should have equal weight. It is certainly the great virtue of sample surveys that 
they permit all voices to be heard, regardless of their origin. This very ability to 
represent all levels of society properly gives the sample survey its superiority over 
less well controlled methods of recording opinion. 


It must be pointed out, however, that the major purpose of many surveys is 
not simply to record a distribution of ‘‘Aye” and “Nay” votes but to provide the 
basis for the prediction of social action. It is clear that in such instances an equal 
accounting of respondents can be misleading in the extreme. This is most appar- 
ent, perhaps, ix. surveys having to do with economic behavior. A survey designed 
to estimate probable expenditures on consumer goods would have to consider 
relative ability to buy as well as wish to buy. A survey to predict redemptions of 
government bonds would need to find out the total number of bonds held by 
each respondent along with his attitudes toward redemption. Taking a less pre- 
tentious example, if a civic group were to survey a neighborhood to ascertain 
willingness to help finance a new swimming pool in the local park it might be 
very badly misled regarding the total amount of financial support that might be 
forthcoming if it did not devise some way to weight the willingness to help by 
the relative ability to help. 


It is not only in surveys of an economic character, however, that considera- 
tion must be given to differences among the respondents who make up the sample. 
Whenever a surveyer attempts to assess public support of a social or political 
issue and to predict consequent social action, he must recognize the fact that the 
attitudes of some members of the population are very much more effective than 
those of others. There are few occasions in our culture when the opinions of all 
citizens have equal social or political effectiveness; indeed, elections present vir- 
tually the only example of such equalitarian accounting. Very much more com- 
monly great differentials of significance attach to the attitudes of different people, 
depending on whether they own newspapers, lead social groups, or otherwise 
hold positions of prominence or whether they belong to the great mass of people 
whose opinions are seldom heard outside their own families. 


Usually the polls ignore these differences in the effectiveness of opinions 
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held by individuals of different status. Thus, they poll the vote on the issue but 
they may give very little indication of the actual pressure of public favor or dis- 
favor the issue has stimulated. This is an important distinction when one recalls 
how few issues in our political system are decided by plebiscite. In most cases 
they are decided by the votes of a small group of elected representatives. The 
votes of these representatives are unquestionably influenced by pressures from the 
public, but pressure in the form of a letter from an obscure citizen cannot be 
realistically equated with pressure from the leading banker, editor, or labor union 
official in the legislator’s district. 


An illuminating example of the difference between popular opinion as 
recorded by the polls and influential opinion as indicated by pressure on Con- 
gress is given in the recent history of the proposal for peace-time compulsory 
military training. Every survey on this issue since the end of the war has shown a 
substantial majority of the public favoring the proposal. However, the pressure 
against the bill from influencial individuals and groups has been prodigious. The 
almost solid opposition of church, school, labor, farm, and other organizations 
has been overwhelming. The bill has lain buried in the military affairs commit- 
tees of the House and Senate for months. The public opinion polls recorded what 
might have happened if there had been a referendum on the issue but they gave 
no indication whatever of what the actual course of the bill would be. 


Surveyers have not as yet devised an effective way of measuring the total 
“head of steam” which particular issues create in the body politic. While it is 
not difficult to weight respondents differentially in regard to their economic status 
there is no easy way to assess the relative influence the social and political attitudes 
of different individuals may have on eventual social action. Economic status is, 
of course, correlated with social influence but this is only a rough relationship 
with many exceptions. 


An approach to one phase of this problem is found in polls of “leaders”, 
surveys which limit their objectives to ascertaining the attitudes of people in 
official positions of leadership. If properly conducted in regard to sampling, these 
studies should be able to give a valid picture of the points of view of labor union 
officials, college presidents, officials of the National Association of Manufac- 
turers, or other special, easily-defined groups of this kind. Such surveys do not, 
of course, reach very far down the ladder of leadership which exists in our 
society and, unfortunately, their function is limited to presenting the attitudes 
of the very group of people whose points of view are most readily predictable 
and most widely publicized. It is hardly necessary to conduct a survey to discover 
how labor leaders stand on the question of wage raises or how N.A.M. officials 
feel about price control. These attitudes are virtually dictated by the position 
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which the official holds. There are, however, many issues toward which attitudes 
are less stereotyped, new problems on which opinion has not yet become en- 
trenched, for example, regarding which leadership surveys may be very informa- 
tive. Leadership polls as conducted at present have the limitation of representing 
only the thin veneer of “big names” who are easily identifiable as leaders, but 
they hold promise as one method of dealing with the problem of weighting 
individual opinions according to their relative social importance. 


A somewhat different type of special survey is the so-called “‘poll of experts”. 
These studies, which purport to present the opinions of qualified experts on 
controversial issues, pose certain difficult questions of procedure which are less 
troublesome in the more conventional type of survey. The first problem in any 
such poll is, of course, who are the experts? In other words, what is the universe 
of the poll? Expertness is in itself to some degree a matter of opinion; the 
surveyer may not find it easy to decide who should fall in his group and who 
should not. If he attempts to avoid such judgmental selection by arbitrarily de- 
fining his universe as including all members of the learned societies or some such 
procedure, he therewith gives equal representation to people whom he knows to 
be unequal in expertness. 


If it may be assumed, however, that it is possible to delimit the universe of 
people who may properly be called “expert” on some question, a second problem 
arises as to how the various groups which make up this universe should be repre- 
sented in the sample. For example, in sampling expert opinion on the question 
of the hereditary or environmental determination of Negro-white differences in 
intelligence test scores, what percent of the respondents should be Negro? Should 
it be 10 percent, the proportion of Negroes in the population? Should it be a 
percentage equal to the proportion of the membership of the learned societies 
which is Negro? Or should it be 50 percent, so that Negro and white opinions 
may be equally represented? In the same manner, what proportion of the experts 
should be psychologists, what proportion biologists? Should they be equal or 
should they be in proportion to the total number of people in these two disci- 
plines? The answers to these questions are important since it is obvious that the 
surveyer's decision regarding them will determine in large part the total picture 
of expert opinion which is obtained. 


Surveys of experts should meet the same requirements of precise methodol- 
ogy to which other types of surveys are held. Judgmental selection of the sample 
is subject to the same criticisms when used in polls of experts as it is when used 
elsewhere. Failure to return questionnaires is just as serious as it is in any other 
mail survey. It is doubtful if any attempt should be made, in reporting expert 
opinions on questions like the one suggested in the preceding paragraph, to throw 
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all of the opinions together into a single distribution. In this example, Negro 
opinion would certainly differ from white opinion, and psychologists would 
differ as a group from biologists. Unless the relative proportions of these groups 
in the sample are determined by an automatic method of random selection from 
the total universe there will be no method of weighting these groups which does 
not depend on the judgment of the surveyer. Polls of experts are both more 
instructive and less subject to criticism when the opinions of the diverging groups 
of experts are presented separately rather than as a total. 


An Opportunity and A Responsibility 


Although most interview surveys are conducted by commercial or non-profit 
agencies which specialize in such work, it seems likely that non-professional fact 
seekers will increasingly turn to this technique for help in their problems of 
information. This trend has in fact already begun. A number of local communi- 
ties have organized local surveys to study the needs and plans of their people in 
order to establish a basis for decisions regarding post-war developments. Usually 
the groups responsible for these surveys have sought the assistance of a surveying 
expert in the more technical phases of the work, but for the most part they have 
proceeded on their own initiative and have achieved very worthwhile results. 


While surveys are most spectacular when done on a national basis with 
hundreds of interviewers and thousands of respondents, their application is by 
no means confined to such large-scale operations. The basic techniques of survey- 
ing are just as applicable to surveys of neighborhoods, club memberships, em- 
ployees, or other small groups. Wherever it is desirable to study the attitudes, 
information, or activities of a group which is too large for complete coverage, 
the procedures of the sample survey recommend themselves. Unfortunately, in 
many such situations these procedures are ignored in favor of less rigorous 
methods. The personnel manager who judges employee morale on the basis of 
reports he receives from those few workers with whom he happens to be on 
friendly terms violates the basic principles on which sample surveys are based. 
The college president who keeps in touch with faculty sentiment through a few 
elder statesmen in the Physics Department commits the same error. Such sources 
are almost always biased, usually in a direction congenial to the inquirer. 


Most surveys assume the voluntary, unpaid cooperation of the respondents. 
If any sizeable proportion of a sample were to refuse to be interviewed, an 
obvious bias would be introduced into the survey findings. Fortunately, most 
Americans are not offended at being asked to give their opinions or personal data 
about themselves. Most respondents only need to be convinced that the survey 
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is a serious endeavor with worthwhile purposes and that their own contribution 
to it will remain anonymous. It is this willingness on the part of the common 
citizen to give information which gives the survey technique its great potentiali- 
ties. If the people of this country were living in an atmosphere of fear and 
suspicion, there would be no possibility of carrying on the great variety of socially 
valuable research which has been done. It is no accident of course that surveys 
have attained their greatest development in those countries in which public opin- 
ion has traditionally been most important in shaping public policy. 


While the survey procedure offers great opportunity to administrators, group 
leaders, social scientists, and other individuals seeking information about people, 
surveys should not be undertaken without a sense of responsibility to the popula- 
tion from which the sample is drawn. In presenting his findings, the surveyer 
is in effect speaking for the group which he has sampled. This is not so impor- 
tant, perhaps, if he is merely reporting their tastes in toothpaste or cigarettes, but 
if he is presenting their points of view on critical social or political issues he has 
assumed a serious responsibility. 


Surveys will prosper only so long as the public feels they are in the public 
interest. If through betrayal of confidences or other unscrupulous use of survey 
findings, suspicion of surveys were to become widespread, the usefulness of this 
technique would be at an end. Nothing is more dependent on voluntary coopera- 
tion. At the present time the American public are for the most part cooperative 
and friendly toward the interviewers who come to their doors. A large propor- 
tion of them appreciate the democratic significance of surveys and are glad to 
participate. It is incumbent upon everyone concerned with survey research that 
this good will be protected and extended. 


Summary 


The articles which comprise this issue make it evident that surveying public 
attitudes is more than just a journalistic enterprise. On the contrary, it is a 
recording of public expression which has the most far-reaching social and poli- 
tical implications. While it is still too early to attempt a full statement of prin- 
ciples regarding survey research it is possible in summarizing the foregoing 
articles to make certain general observations regarding the present status of the 
field. 


1. Interview surveys have now established themselves securely in the 
methodology of social science. The technique which first achieved public promi- 
nence as a method of forecasting election returns is now recognized as one of the 
most versatile and powerful procedures in the entire repertoire of social research. 
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2. Wartime needs for accurate information on public thinking and action 
provided a tremendous stimulus to the development of survey research. New 
techniques and new applications broadened the usefulness of surveys and made it 
possible for them to establish an impressive record of wartime service. 


3. While the war emergency increased the emphasis on survey studies in 
the United States, it had no such effect in Germany or Japan. No true sampling 
of public attitudes and opinion took place in these countries during the war. In- 
dications are that the totalitarian atmosphere is so uncongenial to the basic tenets 
of surveying that the technique cannot even be employed by the ruling hierarchy 
for purposes of pure expedience. The war experience makes it appear that sur- 
veying has an essentially democratic character and cannot prosper where there is 
not a spirit of free thought and expression and where government does not feel 
itself responsible to the people. 


4. Survey research has not taken and cannot take the place of leadership in 
a democracy. With all the undoubted services which surveys can render, it would 
be irrational and even disastrous if our executives and legislators were slavishly 
to follow public opinion in administering the political life of the country. Lead- 
ership and social inventiveness must necessarily come not from the public en 
masse but from its more gifted individual members. It is the genius of American 
democracy that it exploits its leaders and social inventors for the public good. 


5. Survey research has not yet reached the stage of development that its 
findings are typically unequivocal and beyond dispute. The range of possible 
interpretation that attaches to many survey data is much too wide; their meaning 
is often very unclear. In some cases this lack of preciseness is undoubtedly the 
result of carelessness and superficial thinking on the part of the director of the 
survey. To some extent, however, it is a reflection of the fact that survey research 
has not yet come of age. As further improvements in study design and the other 
aspects of survey technique are developed, survey findings will become less and 
less subject to contradictory interpretations. 


6. Survey research is now characterized by a formidable methodology. 
Amateurs who find themselves called upon to conduct surveys are well advised 
to enlist the aid of an expert. Surveys which are poorly done may very easily be 
worse than no survey at all. 


At the present time the most highly developed aspect of surveying pro- 
cedures appears to be the sampling. This is not to imply that all surveys, or even 
most surveys, employ well designed and unbiased sampling methods. It is true, 
however, that the science of sampling has reached the point that it can select 
unbiased samples of known probable error to represent virtually any universe a 
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surveyer is apt to be interested in. In contrast, the interviewing phase of survey 
technology is by no means as well controlled and it is likely that in many surveys 
the interviewing error is considerably larger than the sampling error. 


7. Survey research holds great promise for the future, not only in its large- 
scale contributions, which have been the major concern of this issue, but in its 
applications to the problems of small communities and groups. Surveys are a 
peculiarly American phenomenon. They have flourished in the democratic cli- 
mate of America in a way that one can hardly imagine occurring in any other 
country. They have served the purposes of democracy in narrowing the distance 
between the people and their leaders. There is every reason to believe they will 
play an increasingly useful and significant role in the American scene. 
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The Editor 


Dr. Campbell grew up and went to school on the Pacific Coast. He com- 
pleted his undergraduate work and Master’s degree in psychology at the Uni- 
versity of Oregon, conducting his research on certain social factors affecting 
personality. He completed his Ph.D. in experimental psychology at Stanford 
University in 1936; then joined the staff of the Psychology Department of North- 
western University. In 1939 he received a fellowship from the Social Science 
Research Council to study anthropology at Cambridge University, but war broke 
out the day he landed in London. The last part of his fellowship year was spent 
in the Virgin Islands studying the natives of those islands and attempting to 
relate their personality structure to their present culture and their historical past. 
This study was published in Psychological Monographs under the title “St. 
Thomas Negroes, a Study of Personality and Culture”. The attack on Pearl 
Harbor came shortly after Dr. Campbell rejoined the Northwestern faculty. He 
then joined the staff of the Division of Program Surveys, Bureau of Agricultural 
Economics, U.S. Department of Agriculture, and for the past two years has served 
as Assistant Head. This has involved a close supervisory relationship to a wide 
variety of research projects for governmental agencies. Dr. Campbell states that 
his main concern during these last few years has been in helping develop inter- 
view survey methodology to the point where it can successfully meet the require- 
ments of Government administrators. 


The Other Contributors 


Waldemar Nielsen spent several years before and during the war with the 
Division of Program Surveys. He has recently returned from Japan where he 
worked with the Morale Division of the U.S. Strategic Bombing Survey. Mr. 
Nielsen will edit the next issue of this journal on the results of these studies in 
Japan and Germany. 


Dorwin Cartwright is Assistant Professor in the Research Center for Group 
Dynamics at Massachusetts Institute of Technology. Throughout the war Dr. 
Cartwright was a member of the Division of Program Surveys, working primarily 
on psychological-economic problems of war finance. He is continuing his rela- 
tionship in a consultant role, and also serves in this capacity to the Commission 
on Community Interrelations. 
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Daniel Katz is Chairman of the Department of Psychology at Brooklyn 
College, and Secretary-Treasurer of the Society for the Psychological Study of 
Social Issues. During the war he directed studies for the Division of Program 
Surveys and was on the staff of the Bureau of Intelligence of the Office of War 
Information, directing a variety of projects for the Extensive Surveys Division. 
He also served on the staff of the Strategic Bombing Survey. 


Eleanor Maccoby studied at Reed College and the University of Washington 
before going to Washington, D. C. to do testing and measurement work in the 
State Technical Advisory Service of the Social Security Board. She transferred 
to the Division of Program Surveys in 1942 and is at present a member of that 
staff. 


Robert Holt completed his Ph.D. in psychology at Harvard in 1944 and for 
two years was a member of the staff of the Division of Program Surveys. At 
present he is doing clinical work for the Veteran’s Administration at the Winter 
General Hospital in Topeka, Kansas. 
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